By Gordon Corera
Security correspondent
Russia failed to instrumentality down Ukrainian machine systems with a monolithic cyber-attack erstwhile it invaded this year, contempt galore analysts' predictions. The enactment of a little-known limb of the US subject which hunts for adversaries online whitethorn beryllium 1 reason. The BBC was fixed exclusive entree to the cyber-operators progressive successful these planetary missions.
In aboriginal December past year, a tiny US subject squad led by a young large arrived successful Ukraine connected a reconnaissance travel up of a larger deployment. But the large rapidly reported that she needed to stay.
"Within a week we had the full squad determination acceptable to spell hunting," 1 of the squad recalls.
They had travel to observe Russians online and their Ukrainian partners made it wide they needed to commencement enactment consecutive away.
"She looked astatine the concern and told maine the squad wouldn't leave," Maj Gen William J Hartman, who heads the US Cyber National Mission Force, told the BBC.
"We astir instantly got the feedback that 'it's antithetic successful Ukraine close now'. We didn't redeploy the team, we reinforced the team."
Since 2014, Ukraine has witnessed immoderate of the world's astir important cyber-attacks, including the archetypal successful which a powerfulness presumption was switched disconnected remotely successful the dormant of winter.
By precocious past year, Western quality officials were watching Russian subject preparations and increasing progressively acrophobic that a caller blizzard of cyber-attacks would travel an invasion, crippling communications, power, banking and authorities services, to pave the mode for the seizure of power.
The US subject Cyber Command wanted to observe whether Russian hackers had already infiltrated Ukrainian systems, hiding heavy inside. Within 2 weeks, their ngo became 1 of its largest deployments with astir 40 unit from crossed US equipped services.
In January they had a front-row spot arsenic Russia began paving the mode successful cyberspace for a coming penetration successful which Ukraine's cyber-defences would beryllium enactment to an unprecedented test.
The infiltration of machine networks had for galore years been chiefly astir espionage - stealing secrets - but precocious has been progressively militarised and linked to much destructive activities similar sabotage oregon mentation for war.
This means a caller relation for the US military, whose teams are engaged successful "Hunt Forward" missions, scouring the machine networks of spouse countries for signs of penetration.
"They are hunters and they cognize the behaviour of their 'prey'," explains the relation who leads antiaircraft enactment against Russia.
The US subject asked for immoderate operators to stay anonymous and others to beryllium identified lone by their archetypal names owed to information concerns.
Since 2018, US subject operators person been deployed to 20 countries, usually adjacent allies, successful Europe, the Middle East and the Indo-Pacific region. - though not countries similar the UK, Germany oregon France, which person their ain expertise and are little apt to request oregon privation extracurricular help.
Most of their enactment has been battling state-hackers from China and North Korea but Russia has been their astir persistent adversary. Some countries person seen aggregate deployments, including Ukraine, wherever for the archetypal clip cyber attacks were combined with a full-scale war.
Inviting the US subject into your state tin beryllium delicate and adjacent arguable domestically, truthful galore partners inquire that the US beingness remains concealed - the teams seldom deterioration uniform. But increasingly, governments are choosing to marque missions public.
In May, Lithuania confirmed a three-month deployment had conscionable finished moving connected its defence and overseas affairs networks, prioritised due to the fact that of concerns implicit threats from Russia successful the aftermath of the Ukraine invasion.
Croatia hosted the astir caller deployment. "The hunt was thorough and successful, and we discovered and prevented malicious attacks connected Croatian authorities infrastructure," Daniel Markić, the caput of the country's information and quality agency, says.
"We were capable to connection the US a caller 'hunting ground' for malicious actors and stock our acquisition and acquired knowledge," helium adds.
But lukewarm nationalist statements disguise the world that these missions often statesman uneasily.
Even countries allied to the US tin beryllium tense astir allowing the US to basal astir wrong delicate authorities networks. In fact, revelations from erstwhile quality contractor Edward Snowden 10 years agone suggested that the US spied connected friends arsenic good arsenic enemies.
That suspicion means the young men and women arriving connected a ngo are often faced with a stern trial of their diplomatic skills. They amusement up astatine an airdrome hauling dozens of boxes of mysterious method instrumentality and request to rapidly physique spot to get support to bash thing delicate - instal that instrumentality connected the big country's authorities machine networks to scan for threats.
"That is simply a beauteous scary proposition if you're a big nation," explains Gen Hartman. "You instantly person immoderate interest that we're going to spell bash thing nefarious oregon it's immoderate super-secret benignant of backdoor operation."
Put simply, the Americans request to person their hosts they are determination to assistance them - and not to spy connected them.
"I'm not funny successful your emails," is however Mark, who led 2 teams successful the Indo-Pacific region, describes his opening gambit. If a objection goes good they tin get down to work.
Local partners sometimes beryllium with US teams astir successful league rooms observing intimately to marque definite thing untoward is going on. "We person to marque definite we convey that trust," says Eric, a 20-year seasoned of cyber operations. "Having radical beryllium side-saddle with america is simply a large origin successful processing that."
And though suspicion tin ne'er beryllium wholly dispelled, a communal adversary binds them together.
"The 1 happening that these partners privation is the Russians retired of their networks," Gen Hartman recalls 1 of his squad telling him.
US Cyber Command offers an penetration into what the Russians, oregon others, are up to, peculiarly since it works intimately with the National Security Agency, America's largest quality bureau which monitors communications and cyberspace.
In 1 case, impervious of infiltration came successful real-time. One US operator, Chris, who has led aggregate European missions, recalls observing idiosyncratic determination suspiciously astir the machine web of a spouse country.
What was bizarre was that it appeared to beryllium 1 of the section web administrators the squad was moving with. That idiosyncratic was lasting close down Chris. Could it beryllium immoderate benignant of insider threat?
"Is that you?" Chris asked.
"That is my computer, but I curse that's not me," the head responded, transfixed arsenic if watching a movie. Someone had stolen his online identity.
"Finding idiosyncratic connected your web is not a bully infinitesimal particularly erstwhile they are utilizing your credentials," Chris recalls. That infinitesimal conveyed the world of the menace and successful crook helped unafraid much access.
The US teams accidental they stock what they find to let the section spouse to eject Russians (or different authorities hackers) alternatively than bash it themselves. They besides usage commercialized tools truthful that section partners tin proceed aft the ngo is over.
A bully narration tin wage dividends. At the extremity of 1 mission, US operators accidental that section partners handed them a parting acquisition - a machine disc containing malicious software, oregon malware, from different web the squad had not been inside.
Each ngo is antithetic and determination are immoderate wherever an adversary has been recovered connected the precise archetypal time of looking, explains Shannon who has led 2 missions successful Europe. But it often takes a week oregon 2 to unearth much precocious hackers who person burrowed deeper.
A cat-and-mouse crippled is often played with hackers from Russian quality agencies who are peculiarly adept astatine changing tactics.
In 2021, it emerged the Russians had utilized bundle from a institution called SolarWinds to infiltrate the networks of the customers who bought it, including governments.
US operators began looking for traces of their presence. A tech sergeant successful Cyber Command who liked puzzles spotted the mode the Russians were hiding their codification successful 1 European country, General Hartman says. Unscrambling it, helium was capable to found the Russians were hiding connected a network. Eight antithetic samples of malicious software, each attributed to Russian intelligence, were past made nationalist to let manufacture to amended defences.
Hunting is not an altruistic enactment by the US military. As good arsenic providing hands-on acquisition for its teams, it tin besides assistance astatine home. In 1 mission, a young enlisted cyber relation recovered the aforesaid malware they had discovered successful a European state was besides contiguous connected a US authorities agency. The US has often struggled to place and basal retired vulnerabilities domestically, whether successful manufacture oregon government, due to the fact that of overlapping responsibilities betwixt antithetic agencies adjacent arsenic it sends retired its operators abroad.
Hunt Forward missions are classed arsenic "defensive" but Gen Paul Nakasone, who leads some the military's Cyber Command and the National Security Agency confirmed violative missions person besides been undertaken against Russia successful the aftermath of the penetration of Ukraine. But helium and others declined to supply further detail.
This January, the squad successful Ukraine were trying to debar slipping connected icy pavements erstwhile a bid of large cyber-attacks hit. "Be acrophobic and expect the worst," work a connection posted by hackers connected the Foreign Ministry website.
The US squad watched successful real-time arsenic a question of alleged wiper software, which renders computers unusable, deed aggregate authorities websites.
"They were capable to assistance successful analysing immoderate of the ongoing attacks, and facilitate that accusation being shared backmost to partners successful the United States," Gen Hartman says.
The purpose was to destabilise the state up of the February invasion.
By the clip Russian troops flooded implicit the border, the US squad had been pulled out. Knowledge of the carnal hazard for their Ukrainian partners who remained weighed heavy connected them.
Hours earlier the penetration began connected 24 February, a cyber-attack crippled a US outer communications supplier that supported the Ukrainian military. Many predicted this would beryllium the commencement of a question of attacks to instrumentality down cardinal areas similar railways. But that did not happen.
"One of the reasons the Russians whitethorn not person been truthful palmy is that the Ukrainians were amended prepared," says Gen Hartman.
"There's a batch of pridefulness successful the mode they were capable to defend. A batch of the satellite thought they would conscionable beryllium tally over. And they weren't," says Al, a elder method expert who was portion of the Ukrainian deployment team. "They resisted."
Ukraine has been taxable to continued cyber-attacks which, if successful, could person affected infrastructure. But the state it has continued to support amended than galore expected. Ukrainian officials person said that this has been successful portion acknowledgment to assistance from allies, including US Cyber Command and the backstage assemblage arsenic good arsenic their ain increasing experience. Now, the US and different allies are turning to the Ukrainians to larn from them.
"We proceed to stock accusation with the Ukrainians, they proceed to stock accusation with us," explains Gen Hartman. "That's truly the full thought of that enduring partnership."
With Ukrainian and Western quality officials expressing concerns that Moscow whitethorn respond to caller subject setbacks by escalating its cyber-attacks, it is simply a concern that whitethorn inactive look further tests.