WASHINGTON >> The FBI has been reviewing the handling of thousands of terrorism-related tips and leads from the past three years to make sure they were properly investigated and no obvious red flags were missed, The Associated Press has learned.

The review follows attacks by people who were once on the FBI’s radar but who have been accused in the past 12 months of massacring innocents in an Orlando, Florida, nightclub, injuring people on the streets of New York City, and gunning down travelers in a Florida airport. In each case, the suspects had been determined not to warrant continued law enforcement scrutiny months and sometimes years before the attacks.

The internal audit, which has not been previously reported, began this year and is being conducted in FBI field offices across the country. A senior federal law enforcement official described the review as an effort to “err on the side of caution.”

The audit is essentially a review of records to ensure proper FBI procedures were followed. It’s an acknowledgment of the challenge the FBI has faced, particularly in recent years, in predicting which of the tens of thousands of tips the bureau receives annually might materialize one day into a viable threat.

Investigations that go dormant because of a lack of evidence can resurface instantly when a subject once under scrutiny commits violence or displays fresh signs of radicalization. FBI Director James Comey has said it’s like finding a needle in a haystack and determining which piece of hay may become a needle.

Though there’s no indication of significant flaws in how terrorism inquiries are opened and closed, the review is a way for the FBI to “refine and adapt to the threat, and part of that is always making sure you cover your bases,” said the law enforcement official, who was not authorized to discuss the matter by name.

The pace of the FBI’s counterterrorism work accelerated with the rise of the Islamic State group, which in 2014 declared the creation of its so-called caliphate in Syria and Iraq and has used sophisticated propaganda to lure disaffected Westerners to its cause. By the summer of 2015, Comey has said, the FBI was “strapped” in keeping tabs on the group’s American sympathizers and identifying those most inclined to commit violence.

Social media outreach by IS has appealed to people not previously known to the FBI but also enticed some who once had been under scrutiny to get “back in the game,” said Seamus Hughes, deputy director of George Washington University’s Program on Extremism.

“The fact that there was a physical location and a caliphate announced, it helped kind of drive folks back in when they might have drifted away,” Hughes said.

The review covers inquiries the FBI internally classifies as “assessments” — the lowest level, least intrusive and most elementary stage of a terror-related inquiry — and is examining ones from the past three years to make sure all appropriate investigative avenues were followed, according to a former federal law enforcement official who spoke on condition of anonymity to discuss the process.

Assessments are routinely opened upon a tip — whether from someone concerned about things such as activity in a neighbor’s garage, a co-worker’s comments or expressions of support for IS propaganda — and are catalogued by the FBI. The bureau receives tens of thousands of tips a year, and averages more than 10,000 assessments annually.

FBI guidelines meant to balance national security with civil liberties protections impose restrictions on the steps agents may take during the assessment phase.

Agents, for instance, may analyze information from government databases and open-source internet searches, and can conduct interviews. But they cannot turn to more intrusive techniques, such as requesting a wiretap or internet communications, without higher levels of approval and a more solid basis to suspect a crime or national security threat. The guidelines explicitly discourage open-ended inquiries and say assessments are designed to be “relatively short,” with a supervisor signing off on extension requests.

Many assessments are closed within days or weeks when the FBI concludes there’s no criminal or national security threat, or basis for continued scrutiny.

The system is meant to ensure that a person who has not broken the law does not remain under perpetual scrutiny on a mere hunch that a crime could eventually be committed. But on occasion, and within the past year, it’s also meant that people the FBI once looked at but did not find reason to arrest later went on to commit violence.

In the case of Omar Mateen, that scrutiny was extensive, detailed and lengthy.

Mateen, who shot and killed 49 people at an Orlando nightclub in June, was investigated for 10 months in 2013 and interviewed twice after a co-worker reported that Mateen had claimed connections to al-Qaida.

As part of a preliminary investigation, agents recorded Mateen’s conversations and introduced him to confidential sources before closing the matter. That kind of investigation is more intensive than an assessment and permits a broader menu of tactics, but it also requires a stronger basis for suspicion. Mateen was questioned again in 2014 in a separate investigation into a suicide bomber acquaintance. Comey has said he has personally reviewed that inquiry’s handling and has concluded it was done well.

The FBI in 2014 also opened an assessment on Ahmad Khan Rahimi, who last September was charged in bombings in Manhattan and New Jersey, after his father expressed concerns about his internet use and his associations. The FBI said it closed the review after checking databases and travel and finding nothing that merited continued investigation.

Esteban Santiago, the man accused in the January shooting at the Fort Lauderdale, Florida, airport that killed five people, had also been looked at by the FBI. He had walked into the bureau’s office in Anchorage, Alaska, two months earlier and claimed his mind was being controlled by U.S. intelligence officials. In that case, too, the FBI closed its assessment after interviewing family members and checking databases.

Each act of violence has raised questions about whether the FBI missed signs or should have been more aggressive in its investigation. With thousands of assessments pouring in, those decisions aren’t easy.

“If you’re looking at all the cases, if everything’s blinking red, you have to make a judgment call every time,” Hughes said.