WASHINGTON » The March 2004 confrontation in the hospital room of Attorney General John Ashcroft that was a dramatic high point in the internal debate within the Bush administration over warrantless surveillance was apparently set off by a secret National Security Agency program. And that program was vacuuming up "metadata" logs of Internet communications, according to a draft of a 2009 NSA inspector general report obtained by the British newspaper The Guardian.

The report, the latest document given to the paper by former NSA contractor Edward J. Snowden, might clear up a long-running mystery over which program White House officials wanted Ashcroft and other Justice Department officials to sign off on when they went to his Washington hospital room. Because of their refusal, according to the report, the Bush administration shut down the metadata collection for several months, then re-established it under a secret order from a national-security court established by the Foreign Intelligence Surveillance Act, or FISA.

The program continued to operate for the first two years of the Obama administration but has since ended, Shawn Turner, a spokesman for the office of the director of national intelligence, said Thursday.

"The Internet metadata collection program authorized by the FISA court was discontinued in 2011 for operational and resource reasons and has not been restarted," Turner said. "The program was discontinued by the executive branch as a result of an interagency review."

A separate NSA program that has been collecting domestic "telephony metadata" – logs of all telephone calls dialed by Americans – has continued. That program was among the first of Snowden’s revelations. It is not clear whether Internet metadata collection has continued under a different program.

The NSA uses the metadata to analyze links between people in an attempt to identify networks of suspected terrorists, a process the report calls "contact chaining." The inspector general report says they typically "analyzed networks with two degrees of separation (two hops)" from the known suspect to create investigative leads submitted to the FBI and the Central Intelligence Agency in reports they called tippers.

The content of the inspector general report – a history of the Bush administration’s warrantless surveillance programs, which eventually came under court oversight – was first reported by The Washington Post.

Bush’s original surveillance program, established Oct. 4, 2001, had four components. It sucked up the contents of telephone calls and emails, as well as their "metadata" logs. This was done without court orders and outside statutory regulations and was based on closely held legal memorandums; even the NSA’s general counsel was not allowed to see them at first, according to the report, which invoked the president’s inherent powers as commander-in-chief.

Over time, however, there was turnover at the Justice Department, including the arrival of a new head of its Office of Legal Counsel, Jack Goldsmith, and a new deputy attorney general, James Comey, who last week was nominated by President Barack Obama to be director of the FBI. In December 2003, Goldsmith began to object to the legality of some aspects of the programs, and he persuaded Comey – who took over temporarily as attorney general when Ashcroft became ill the following March – not to sign a document reauthorizing it. The report says that the Justice Department’s concerns were focused on the Internet metadata program, which was then based on a theory that the NSA did not "acquire," for legal purposes, the bulk communications when it collected them but instead only when human analysts selected certain data to examine because they met certain criteria.

On March 19, 2004, after taking into account the objections of Comey and his allies, Bush rescinded his authorization to the NSA to collect bulk Internet metadata and gave the agency a week to stop collecting it and to block access to its existing database, according to the report.

During the same month, the report says, the Bush administration also ended the secret domestic surveillance programs against Iraq it had begun the year before under a theory that Iraqi spies were "engaged in terrorist activities and presented a threat to U.S. interests in the United States and abroad." The credibility of Bush administration claims linking Iraq to al-Qaida and other terrorist groups was, at the time, a subject of deep controversy. It is not clear whether ending the program was a coincidence or connected to the Justice Department’s rebellion against the White House over the administration’s legal theories about surveillance.

After Bush’s order to end the collection of Internet metadata, administration lawyers looked for a new legal basis to restart the program. Under this theory, the Foreign Intelligence Surveillance Court could authorize such bulk collection under a "pen register/trap trace" statute, which traditionally allows law enforcement officials to keep logs of communications sent and received by particular phone numbers or email accounts.

The chief judge of the national-security court, Colleen Kollar-Kotelly, issued an order July 15, 2004, authorizing the resumption of the program, according to the report. None of the other judges on the court were apparently told about the programs or the new order; this was the first time one of the NSA’s secret surveillance programs had come under the court’s oversight and authority.

In 2006, according to the report, NSA was faced with another legal challenge when The disclosed the existence of the warrantless surveillance program. In response to the disclosures, one of the telephone companies that was secretly providing its customers’ data to the NSA on a voluntary basis asked for a court order compelling it to comply to protect itself legally, forcing the Bush administration to develop new legal theories to support other surveillance.

That May, the national-security court issued its first order to the telephone companies to log metadata from phone calls, citing a provision in the Patriot Act that allowed the government to obtain business records deemed to be "relevant" to a counterterrorism investigation.

It was apparently more difficult to get the court’s agreement on a legal justification for the collection of the contents of phone calls and emails than of Internet metadata. But the Justice Department came up with a theory that the word "facility" in the Foreign Intelligence Surveillance Act, which had traditionally been understood to mean a specific telephone number or email address, could be "changed to encompass the gateway or cable head that foreign targets use for communications" – that is, the entire line. That gave the NSA access to the contents of a huge number of communications. In January 2007, a judge on the FISA court issued two orders, one covering the collection of foreign communications and another dealing with domestic ones. After the contents were collected in that fashion, rules would be applied to screen out Americans’ communications in some circumstances and setting limits on when the database could be consulted, the report said.

© 2013 The New York Times Company