Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Finally a named source, but still no photos and the alleged hacked board is still not in the hands of a public security researcher.

The "trojan ethernet connector" paragraph mentions similarity to an NSA implant, which appears to be this: https://en.wikipedia.org/wiki/NSA_ANT_catalog#/media/File:NS...

I'm now wondering if someone found an NSA implant and misreported it as Chinese. We're going to end up in the stupid situation where people are afraid to report foreign intelligence attacks because it's illegal to report an attack by US intelligence agencies, aren't we?



A named source, but not a named victim, in this case. I would not call this verification.

This is a really hard story to know what to think about. On the one hand, yes, hardware implants are a major risk. And having so many of our electronics manufactured in a country with massive state control over its economy and with which we have an adversarial political relationship is definitely a big concern.

On the other hand, the denials from the companies cited in the first article are remarkably strong. And again this article fails to give relevant details. It just cites a security contractor who says he had a client who had this issue.

But Bloomberg is a serious news organization and they are holding strong on this story as well. So what to think?

It strikes me that if your goal was to ramp up tension between the US and China at multiple levels, then planting this sort of story would be a great way to accomplish it. Politicians can cite national security. Wary consumers are triggered over privacy. Corporations become more and more gunshy of investing in China and partnering with Chinese manufacturers.

I hate to dream up conspiracy theories. And yet, we live in a world where many states, politicians, organized crime groups, political groups, and corporations are all intentionally spreading disinformation of all sorts all the time designed precisely to ratchet up tension and suspicion.

I don't really believe that's what's going on just yet. But I also don't believe it's as straightforward as the Bloomberg stories make it out to be, either. Something very strange is going on.


Yeah - when I add to it that, as a non-American, I can (annectodaly) observe a rise in different kinds of news that involve China in a negative context for the last 6m especially, it's hard to form an opinion.

In terms of security concerns also - come on, we know by now to which lengths the US goes in this area, and they're surely doing worse stuff than this, I'd expect no one would doubt it any more. So, either they are genuinely surprised by this, which would be silly (a politically adversarial nation using an obvious opportunity - cheaper stuff being produced there for decades - and doing the same), or it's a part of a broader narrative that's being built.

And, to be clear, I don't think we (the world outside China) shouldn't be a bit worried given in what position _we've_ put China and how strong they are now - it's just that this kind of mass-manipulation and propaganda is the most-detested way of doing it for me...


> I don't think we (the world outside China) shouldn't be a bit worried given in what position _we've_ put China and how strong they are now

At least China won't be as dangerous as the US or the Soviet Union in that it has absolutely no interest in enforcing its political ideals on other places or becoming a world police. I see no real reason why people should be worried about the rise of China as if this will turn the whole world Orweillian. China has its own way of organizing the vast and complex country and it's hard to come up with better practical solutions. Still if one doesn't like it they can just live somewhere else, and the authorities don't care, as long as it doesn't hurt Chinese business.


> .. no interest in enforcing its political ideals on other places or becoming a world police

Not yet. But becoming a banker for whole third-world is almost done. And when your debts are big, you lose sovereignty.


Still doesn’t mean you shouldn’t be concerned by China’s super position in the global supply chain...


You should, but China rarely cares about people beyond its own boarders. They don't have the power of the U.S. to reach for anyone across the globe, so I think NSA doing this is a tad more worrisome.


I think the main difference between the US and China, the Chinese have no mission to convert every country to their thinking. Not that they influence countries through their investments, see Greece as an example.


Ummm other than Silk Road pulling tons of countries into their sphere, using it to push new Chinese standards for people to switch towards, making HK/Taiwan/Tibet fully incorporated and culturally homogenous, taking islands away from other countries across SE Asia and calling it all china’s sea, etc etc.

They’re just more subtle, until they’re not. They have a very long term outlook on their efforts.


China does have ambitions, sure, but they're mostly local ones. I was mainly talking about cases like Kim Dotcom, where the U.S. reached all the way to NZ. China doesn't have Five Eyes, has not attacked a foreign country etc. It does not mean this cannot change, but I get the feeling China strives for a more local "sphere of influence" strategy, (hence the things like you mentioned), whereas the US strives to maintain "global dominance". As someone sitting in Western Europe, I don't think China has that much interest to dominate here.


China is not pushing countries to become 'communist' states.


You need to be a citizen of a nation neighbouring China to know the severe pressure that China exerts on borders, trade and geo-politics. You just don't hear about it much in the world-news as it doesn't affect the US.


China is not pushing countries to become 'communist' states.


"... don't ..."


They kidnap people from other places who are dissidents including from Hong Kong, Malaysia, and even the us.


Here's an article discussing this worldwide kidnapping. https://foreignpolicy.com/2018/03/29/the-disappeared-china-r...


This claim seems like a big dilemma for US white-hat security researchers:

1. As a white-hat security researcher, you have an ethical responsibility to publicly disclose vulnerabilities after doing the necessary due diligence (informing the affected parties privately, and giving them the necessary time to respond, investigate, and come up with an acceptable solution).

2. As a US citizen, you can't report attacks carried out by US intelligence agencies.

I can definitely see the responsibility that patriotic duty would entail, but a citizen with no links to their country's intelligence agency being held responsible for the said agency's failure in maintaining operational discretion doesn't seem sensible to me.

Edit: update formatting.


> As a US citizen, you can't report attacks carried out by US intelligence agencies.

Who says? Unless you've received a National Security Letter, a gag order from a court, or have a pre-existing relationship with the government that governs disclosure (e.g. security clearance), there's nothing preventing a researcher from disclosing lawfully obtained information. Stumbling upon a secret investigation doesn't make the information unlawfully obtained, even if you suspect it might be a secret investigation.


Are you sure that the Espionage Act (1917) doesn't cover this? In Australia we have many recent laws that completely restrict our ability to whistleblow on any government issue (though it's not illegal if we ensure that non-Australian nationals know about it -- which is obviously an impossible and stupid standard).


It might (particularly subsections (d) and (e))[1], but only because the wording is so broad. Whether such an application would be legal is another matter. I suspect it would not absent specific intent (i.e. you're deliberately seeking out secrets to share) or a duty (security clearance).

I suppose intent could be there if you share information about a device that says, "Warning: national defense injured if you disclose". But absent a duty I don't think a court would impute intent, especially considering the Free Speech issues (somewhat peculiar to the U.S.).

Notice that nobody has seriously suggested (AFAIK) that the journalists who assisted Snowden should be charged under the Espionage Act, even though their acts would seem to fit multiple provisions. I think that's because unlike Snowden they had no duty, which means the bar for the requisite intent and knowledge (i.e. whether something is really going to harm national defense) is incredibly high.

But who knows? It's a good point and it poses a ton of questions. Still, personally if I found a spying device on something I wouldn't hesitate to disclose it if it seemed noteworthy. I wouldn't feel chilled by the Espionage Act. The same law in some other country? Probably would think twice.

[1] https://www.law.cornell.edu/uscode/text/18/793


> As a US citizen, you can't report attacks carried out by US intelligence agencies.

Sure you can. Short of a gag order (and maybe not even then) you can report intrusions all you like.

In any event, how does one determine the nationality of hardware that shouldn't be there? It's not like there's going to be a snarky "Designed by the NSA in Fort Meade" logo on the chips in question.


If (IF!) you hold a civilian or military clearance, then you have a legal Duty to Report (DTR). That holds true whether its data in your clearance level or not.

You also abide by a whole slew of laws regarding sensitive, secret, top secret, or SCIF information. If I knowingly, or even suspect, some information if classified, and I transmit it to anyone else than my federal assigned contact, I'm breaking major federal laws.

A lot of security professionals in the US have such clearances. So finding a NSA implant or such proof makes it dangerous to talk about by default.

So yeah, a gag order by profession.


Are you really a white-hat if you have to disclose vulnerabilities to an organisation known to exploit (or at least hoard) them?


Vulnerabilities and weaponized implants are very different things.


Please quote me where I claimed to be a white hat.


You didn't, but the context was:

> This claim seems like a big dilemma for US white-hat security researchers

It seems like the two are mutually exclusive.



> But Bloomberg is a serious news organization and they are holding strong on this story as well. So what to think?

Are they? The authors of this story published an unverified and in corroborated story about Heartbleed a few years ago, claiming that the NSA knew about it and was exploiting it (https://www.washingtonpost.com/blogs/erik-wemple/wp/2014/04/...).


And here's someone else calling out Bloomberg for an "unethical hatchet job" when reporting on a technical issue: https://www.semiaccurate.com/2012/10/08/bloomberg-wrong-abou...


To be fair, Clover Trail had all sorts of driver issues that never got resolved. I personally had to deal with the shitty GPU drivers for work. I can't speak to the power management since we were using the chip in a place where power management didn't matter, but I can see those being shit too.


I think the issue is that they're a serious news organization about some subjects, and a dumpster fire for others. It's hard for the general public to recall which subjects they are authoritative on and which not.


It would not surprise me if the NSA did know about it but it's a shame they didn't have any proof.


It seems that the author of the research is not happy with Bloomberg's spin on the story: https://www.servethehome.com/yossi-appleboum-disagrees-bloom...


> But Bloomberg is a serious news organization and they are holding strong on this story as well. So what to think?

When the articles published by those "serious news organizations" concerns China, disinformation / lack of evidence are really common place if you carefully examine their source. I used to do so from time to time but grew tired of that


That's not a conspiracy theory. It's standard type of policy and it would be surprising otherwise.


If information, ideas, knowledge were shared openly we wouldnt have these kinds of ridoculous events. This kind of news is what keeps nations siloed and prevents collaboration. At the same time maybe this will also force us to abandon trust all together and move towards verifying.


Indeed. I saw a brilliant presentation in 2012 by Michael Mitzenmacher from Harvard on verifiable computing in the cloud. It was based on this paper:

https://arxiv.org/pdf/1202.1350v3.pdf


I think we should judge by facts,not by stereotypes. In my opinion, newspapers are all propaganda machines driven by some their benefit.


> I think we should judge by facts,not by stereotypes.

Cool. Let us test.

Error: Line 2:

> In my opinion, newspapers are all propaganda machines driven by some their benefit.


> I hate to dream up conspiracy theories.

No conspiracy theory needed, traditional media is dying, and in their last gasps of air they are destroying their credibility for the sake of clickbait articles without proper facts and coberation. They are being deceptive, because they know outrage and politically dividing stories are still working.

It's really said, but transparent. No way, Apple officially writes that rebuttle on their website if the story is true.


German telecom employee here. I've seen a number of sneaky backdoors and intercepting devices at all levels in my career. The most interesting thing was a server where TCP connections that were about to close (TCP FIN) were suddenly intercepted to dump additional (encrypted) data that was't part of the original flow. Obviously there was something out there that was seeing both sides of the flow and intercepted parts of it. We successfully confirmed the problem was on our (in)side by booting the affected server on a USB stick and made it generate controlled traffic to controlled destinations on the internet that were synchronized using a LFSR. The server was decommissioned and the issue was escalated above my paygrade with clear instructions not to talk about it. I won't give an exact year for this incident but it happened in this decade but before Snowden.

Personally I'm confident Bloomberg's reporting is accurate to a high degree. Based on prior experience with investigative journalism there's no way they would go all in with a story like this if they weren't standing on firm ground. Every single sentence would've been vetted. For each statement made there would be someone whose job would be to reject it unless you could back it up properly. This is also why you don't see these entities defending their story against random criticism that pops up. Most if not all decisions have already been made by the time it goes public.

The fact that there's now a second story on the same topic is a good sign. The reporting of these things are usually followed up by additional pieces to increase the impact (and revenue of course).

They claim they have 17 independent sources. That's pretty impressive in itself. It also means that they probably worked real hard verifying their sources' claims and inputs. I find it unlikely that they would've acquired all those sources unless the thing was real.


> Based on prior experience with investigative journalism there's no way they would go all in with a story like this if they weren't standing on firm ground. Every single sentence would've been vetted.

And based on my prior experience I would make the exact opposite conclusion. Technical writers are rarely technical, and they seem to be happy to make stuff up and mislead - even if unintentionally - so long as they make their deadlines.

Every tech article written about a subject I was directly involved in has not even gotten the spirit of the topic remotely accurate, much less minute technical details such as what chip is used where.

That said - I fully believe supply chains are entirely compromised. I just don't think in this case I'd really put much stock into this specific reporting - they've already been caught blatantly misleading their readers by putting up a photo of a stock Mouser part and not denoting it as such.


I agree that technical details sometimes gets misrepresented or come out plain wrong. That's my observation as well and it's annoying when you're knowledgeable in the subject and try to make sense of what you've read (or read between the lines).

I think a contributing factor is that it's generally hard to write about things you don't fully understand with the correct nomenclature. Especially when you might not be able to talk/ask for help about specifics with people more knowledgeable because of the secretive process.

Things could've been dumbed down, intentionally or unintentionally, by those involved. It wouldn't be hard to imagine a conversation like: "-So it was sort of a coupler thing? -You could say that, yes". Or what if the technical detail came from a Chinese source and Google translate mangled it?

The coupler thing is dumb and so is the picture (assuming it was a random product picture) but at least they might serve as a way of communicating the big picture: a hard to spot electronic "coupler" thing.


Not in this case. One of the sources was interviewed and said the technical details were only suggestive: https://risky.biz/RB517_feature/

Maybe there is something there, and/or there is a reason to talk/substitute in vague terms, but insofar as the explicit technical details are concerned, they don't appear credible. Then you're left with an empty allegation that you will have to decide to believe or not based on no other ground than potentiality.


Interesting experience and thank you for the first-hand perspective!

I do have to disagree about the competency of Bloomberg, though, they publish a lot of speculative, low-tech AI/ML scare articles that can be described at best as "inaccurate" and, more realistically, as "making stuff up". They used to have a good reputation, probably from their financial journalism, but their tech work is not good but any reasonable measure, in my personal opinion.


There is a big difference between claiming 17 sources, a claiming all 17 sources corroborate the full story. The Apple letter to Congress highlight that Bloomberg is relying on a single source for the specific claims about compromised servers being found at Apple.


I am not sure if you're just a German telecom employee based out of US or an actual native German working for this German telecom company in Germany. If latter, your english is extraordinarily above and beyond what I've generally exhibited with a lot of my personal German friends. Just a naive observation so please don't take this in any wrong or defamatory manner.


The lack of (publicly available) evidence is annoying, since there are a lot of people who'd love to check their own servers. As this is an attack directed at high profile targets it's unlikely the average size company will have ended up with one of those, but it's still a fun exercise IMO.

It would also be great to know what the attribution is based on. Just the fact that they're manufactured in China? Who else might get their hands on these devices in the shipping chain? What kind of traffic did they monitor? I guess just observing it's talking to a Chinese address doesn't tell much. I mean, just take an S3 bucket and dump your stuff in there. Setting up your own server in your home country pretty much screams "we're here!"

> I'm now wondering if someone found an NSA implant and misreported it as Chinese. We're going to end up in the stupid situation where people are afraid to report foreign intelligence attacks because it's illegal to report an attack by US intelligence agencies, aren't we?

That's pretty tinfoily, but it'd be a cool way to still report on it. "Whoopsie, I totally thought it wasn't you guys, sorry for disclosing"


"Who else might get their hands on these devices in the shipping chain?"

From the original Businessweek article:

"Supermicro has assembly facilities in California, the Netherlands, and Taiwan, but its motherboards—its core product—are nearly all manufactured by contractors in China."


"Supermicro has assembly facilities in California, the Netherlands, and Taiwan, but its motherboards—its core product—are nearly all manufactured by contractors in China."

That's interesting. As someone who has bought hundreds of thousands of dollars of gear from Supermicro (and has been a huge fan of their products and designs) I always thought their chassis were their core product.

Recently SM started to go down the "you can't buy our JBOD chassis without buying them full of our qualified drives" ... I knew that was the end of the golden age (of SM).

Luckily this coincided with the introduction of the 60bay HGST JBOD chassis. We haven't looked back.


> Luckily this coincided with the introduction of the 60bay HGST JBOD chassis. We haven't looked back.

Yes, these units are stellar and anyone buying Supermicro JBOD units should be looking into these as much better replacements. If you have volume they can be even more competitive than Supermicro if you push.


One very, very small gripe is that the HGST JBODs have no power switch. You power them on and off by inserting or yanking the power cables. Not my favorite SOP ...


Is that a real thing? Holy cow, I'm shocked (bad pun intended). What about adding an inline switch in the cord? Unless they expect everyone to be using a managed power system where each plug can be turned on/off, this just seems very odd decision to make.

<off to look up this beast>


As someone that flipped the power switch on a rack mounted machine by accident before, I could see how a power button or switch would be consitered a liability more than a benefit, especially when the solution (pull the power cable) is simple, foolproof, and doesn't happen that often to warrant optimizing!


Every storage vendor eventually goes full NetApp. The money just looks too good.


I have to assume we'll start to see a rise in American high tech manufacturing for security purposes alone. Some of these companies may want to manufacturer these critical components themselves, maybe even hand deliver them from their US factory to their customers in the US too.

I know that some refineries do direct delivery for some of their large customers, especially industrial lubricants and other by-products. If the order is big enough, or someone wants to pay the premium, then direct delivery could be very feasible for tech too.


It's odd to me to assume that people should trust US-based supply chains. We know that the NSA has done supply chain attacks in the past[1], while in this case we only have allegations of China doing the same (don't get me wrong, I wouldn't be surprised if China did this, I'm just saying we have more evidence for the NSA doing it).

Personally, as someone outside the US, I would gladly trust alleged Chinese malware over known NSA malware. Or even better, literally any other country outside the 5-eyes.

[1]: https://www.theguardian.com/books/2014/may/12/glenn-greenwal...


Is there any way to solve this problem without needing a "trusted manufacturer"?

I know it won't probably won't apply to general purpose motherboards or devices, but is there a way to design or build some components or devices in a way that you can verify that they can perform their purpose and nothing more?

If we start with that concept, and slowly build up "verifiably secure" components, they can be the islands of security that we can build off of without having to worry if the manufacturing plant left their door open one day and some random person was able to sneak in.


What happens when your attacker knows how your safeguards work and can route around your door though the windows?

For a motivated and well funded attacker who has an ability to manufacture a replacement chip with an additional coprocessor that can siphon or modify data from the main processors, network cards, and baseband modems, short of decapping every chip and component that comes through your assembly line your resources would be better spent on establishing trust mechanisms with your suppliers and the transportation couriers touching your devices before the end user acquires it.

https://en.wikipedia.org/wiki/Tailored_Access_Operations#Kno...

http://www.spiegel.de/international/world/the-nsa-uses-power...


My thought was it would be something that would get more secure the more knew about it, similar to math proofs or cryptography code.

A way to verify a chip is working as expected in a way that it can't be gamed without breaking multiple fundamental proofs, so that you won't need to worry as much about who makes it, just that it "passes the tests". (and you'd probably need a system to validate the validators, but splitting up the people involved means it is significantly harder to hack multiple products to all have them falsely verify each other)

Obviously I have no idea what I'm talking about and am just kind of musing at the idea, but trying to secure the whole supply chain from digging materials out of the ground all the way until it is in the hands of the consumer seems like an exercise in futility. You'll never be able to secure it in all cases, and like you said a truly motivated attacker is going to be able to break the chain (even if it means threatening a handful of people with death so you can get 5 minutes alone with a board).


What high technology manufacturing America does is in the security space, otherwise Japan is a trusted source.


>we'll start to see a rise in American high tech manufacturing for security purposes alone.

Already exists in the form of 'country of origin' procurement for high security applications.


I've been looking in detail at three different Supermicro motherboards but so far have not been able to spot anything. Even against a backlight there is no sign of tampering between the layers.


The most compelling explanation I've heard is that the BMC chip could be programmed by two distinct flash chips, one for factory programming and one for some other purpose. In some SKUs, the latter isn't populated but it has a higher priority than the first chip.

Since there are many flash chips fitting the same pin out, all it took was soldering a compromised flash chip (with firmware for the BMC chip) onto pads that are already part of the design to compromise the whole system without any obvious sign that the board was tampered with (because in some SKUs, both chips were populated).


The BMCs on the newest Supermicro servers are from ASPEED. The X10 models have the AST2400 [0] and the X11 models have the AST2500 [1]. They have ARM CPUs and run, basically, an embedded Linux.

If you wanted to "backdoor" motherboards that shipped with these BMCs, wouldn't it would be much easier to just install your own "customized" version of the firmware on them? It certainly seems that it'd be much more difficult to incorporate another device into the system.

[0]: https://www.aspeedtech.com/products.php?fPath=20&rId=376

[1]: https://www.aspeedtech.com/products.php?fPath=20&rId=440


If I'm right, that's exactly what they did. When the BMC chip boots, it checks two flash chips for firmware so the attacker just uploaded their own code to one of a million standard SPI flash chips and plopped it onto the board. They didn't have to incorporate another device into the system, the system was already designed for two flash memory chips. However, to save money on some SKUs, the manufacturer left one of the positions on the board open.

Normally this wouldn't be worth talking about because most active chips are too complicated and too design/supplier specific to carry out an attack like this, but SPI flash is about as standard a footprint/protocol as you can get in EE short of transistors so if you ship a product that could be reprogrammed from unpopulated pads, you're opening yourself up to a large attack surface.

Honestly, after I read the latest BMC chip theory I was like: "Oh, shit. Have I done that?"


If possible, it is better to have separate hardware that can continuously compromise the firmware. That way your exploit continues to exist even if valid firmware is flashed directly onto the memory module.


well companies like Apple and Amazon are reflashing/updating, so that wouldn't stick.


because all it takes for it to be discovered is someone checking the SPI flash contents


By explanation do you mean theory or is it coming from somebody who has special knowledge of the situation?

I'm not trying to be adversarial, even if it's only a theory it's an interesting one, but given the amount of conflicting information we have regarding this whole mess I think it's important to be clear about what's pure speculation and what's been reported by people supposedly in the know.


I looked at this back in 2013. Here's some slides from a talk I did after spending 48 hours with them[1].

The BMC back then was by a company called ATEN, who make KVMs. The modern BMC is by ASPEED - I don't know if they're related.

What's described in the article is exactly how the old ATEN firmware worked normally. It was a spectacularly poor product from a security perspective.

[1] - http://mandalorian.com/dl/himym.pdf


Maybe you are not a high value target?


That raises an interesting question about just how targeted this kind of attack could be. At manufacture time, do the folks on the assembly line (so to speak) know who a particular board is going to? If not, they would have to add the extra chip to all outgoing boards, which means there should be plenty of them in the wild, no?


If the motherboards were customized for a particular customer, you'd know exactly who they're going to. That would eliminate the problem of letting the exploit travel too widely as well.


Right, but does that happen? I honestly don't know. Clearly a company like Amazon or Apple buys in large enough volume that they could be asking for customized MB's, but does anybody know if that actually happens? If it does, then that would definitely moot the question I was posing above...


Or swap the boards out in transit.


Seems more problematic though. You'd have to manufacture the doctored boards, extract them from the normal shipping process, keep them hidden somewhere, then swap them out for the ones destined for the target customer(s). I guess it could be done, but it seems risky.


Couldn't it be done on-demand? Apple orders X hundred boards, motherboard manufacturer makes their small modification(s) to a line that is currently producing the same models of motherboard as Apple ordered, they produce a handful, then they revert and mix in a few of those modified boards into the real order. I don't really know the exact scale, so maybe they make a few hundred / the entire order with chips in them, but economic cost isn't a big deal for things like this, so even losing money making the modified boards wouldn't be the end of the world (and presumably they get a hefty sum of money for whoever is paying them to do this).

I thought China was famous for extremely short turnarounds for industrial engineering edits, so it seems plausible that they could manufacture the boards in a reactionary way and not need to do much in the way of logistics to get them to their targets.


No comment.


If I was a high value target (and knew about it) I would definitely not let you know, if I was a high value target and did not know about it I would not be able to tell if I was or if I wasn't. So any high value target and anybody else would not be able to tell you they were a high value target.


What about the variation where you're not a high profile target and you know it?


Same here. I have four different Supermicro motherboards purchased in May for servers in my home. I'm sure there exist people and organizations in the world capable of putting malicious hardware on one of these such that I can't detect them. But insofar as I've personally examined them and the available evidence from Bloomberg, color me skeptical...


Ok now try to patch the BMC, you can actually talk to it with openipmi on local host.


I'm familiar with OpenIPMI (I use it for remote fan control a lot) but I'm not clear on what exactly you want me to try?


I really want to see someone on here with access to one of their recent boards try and report on this. I'd try it, but I sold my last Supermicro board years ago.


Try what? Updating the firmware? I do it every time a new firmware version is released.


Back around 2014-2015 supermicro had this bug that would not let you flash the main firmware. Would not happen on every machine maybe 25%. Had to derack and send a number of machines back.


Are your Ethernet shells metal, as described in the article, or plastic which the article describes as normal?


This metal shells rubbish is a key indicator that the whole story is bogus. Metal is completely normal.


You purchased servers from 2013-2015 in May? As in used servers?


You would recognize what looks like an extra resistor?


The supposed infiltrated part is a six terminal RF device. Not something that would ordinarily show up on a server motherboard. In any case, Joe Fitzpatrick has already disclosed that he used the part merely as an example and Jordan Robertson expanded that into a work of fiction.


I hadn't seen this before, but searching for "Joe Fitzpatrick Jordan Robertson" finds https://appleinsider.com/articles/18/10/08/security-research... which seems to be what you were referring to?


The original source is Joe Fitzpatrick's interview with the Risky Business infosec podcast. Apple Insider is just summarizing some of the points from that interview:

https://risky.biz/RB517_feature/


Where is the 6-terminal claim from?



There were quite a few pictures of what is supposed to be the device in the Bloomberg article. Knowing what they say it looks like and knowing roughly where to look I'm 99.9% sure that none of the boards I have here have that device on them.


I don't have the reference handy but someone claimed to be a source and they pointed to a generic item on digikey / mouser as an example. I imagine that it got extrapolated by Bloomberg into that.

They really have no idea what they are talking about at this time and it's probably fluff.


I'm not sure why you're downvoted, except the lack of citation. Your recollection is correct, it's from the Joe Fitzpatrick interview with Risky Business, which was quoted by Apple Insider. (Fitzpatrick was named as a source in the original Bloomberg article.)

Long story short, that photo does not show the device involved.

"Robertson was unable to produce photographic evidence of the chips in question, saying they were described to him by protected sources. Indeed, Robertson in September asked Fitzpatrick what a "signal amplifier or coupler" looks like, suggesting the publication narrowed the attack package down to that particular component. Fitzpatrick sent Robertson a link to a very small signal coupler sold by Mouser Electronics. "Turns out that's the exact coupler in all the images in the story," Fitzpatrick said.

https://appleinsider.com/articles/18/10/08/security-research...


The image caption on the bloomberg story reads "Microchips found on altered motherboards in some cases looked like signal conditioning couplers". They didn't claim "that's the chip"


It has more terminals that a resistor, it's a pretty unusual package and it would stand out enough for me to spot it knowing that it is there. The area of the PCB that you could expect that thing to live in is about 5x5 cm square.


Isn't the idea that the boards weren't tampered with but manufactured by contractors including extras?


Well, that depends on your definition of tampering, but if you want to exclude manufacturing something that is not what was specced then I am fine with that but please do supply a new term.

I would definitely spot that device if it were on these boards because it was described in detail and there were some pictures of what it supposedly looked like.

A device like that is not on either side of the board and it isn't in between the outer board layers (where it would be much harder to spot, especially if the cavity would be covered by a ground plane on one side).

I am not saying it is impossible, it is just very hard to hide something like that once you know it is there. The only candidate spots left that I can not check without destruction is underneath some of the devices or inside some of the devices. That would be a different level of sophistication than the original article alluded to.


> I would definitely spot that device if it were on these boards because it was described in detail and there were some pictures of what it supposedly looked like.

In case you missed it, there is an article posted today [0] that has this quote from "Hardware security expert Joe Fitzpatrick", one of the Bloomberg sources, regarding "the supposed spy chip":

> In September when he asked me like, “Okay, hey, we think it looks like a signal amplifier or a coupler. What’s a coupler? What does it look like?” […] I sent him a link to Mouser, a catalog where you can buy a 0.006 x 0.003 inch coupler. Turns out that’s the exact coupler in all the images in the story.

[0]: https://9to5mac.com/2018/10/09/bloomberg/


Oh, that's interesting. So they basically took one guys hypothetical and turned that into a news item positively seeded with images of the hypothetical, rather than an actual device.

The original article has now dropped into the real of SF for me until they show a detailed shot of an actual board with a parasitic device on it. Until then this is a wild goose chase.

Thank you for pointing this out.


My understanding is that certain parts on the PCB were swapped out for malicous parts. If that's the case, it's probably not something that could be uncovered by a purley visual inspection. The 'spy' chips were likely made to look identical to the original parts.


That’s not what the BW/Bloomberg story claimed - it specifically called out a chip that wasn’t on the official BOM and had been added to the build.


The initial allegations from Bloomberg suggested ON the otherboard, not in, as I understand it.


There was mention of one being discovered buried inside the FR4 PCB material.


I don't think you'll find this in a board that doesn't otherwise normally have lots of other buried components ... The added cost of that extra process (using buried components) is so way higher than normal and such a board is going to look noticeably different from a normal board ... I'm tempted to think that someone told the Bloomberg guys that it was possible and the took it that it had happened


Yeah, particularly given it was against a US telecom company, the NSA would make sense as the source of the implant.


No that would make 0 sense. The NSA doesn't "attack" american companies with covert implants.

They get FISA court orders that force american companies to attach their equipment.



> ...doesn't "attack" american companies with covert implants.

Specific example aside, it's worth talking about why this does happen. "Black bag jobs" can mean "we didn't get a warrant", but they can also mean "we got a warrant and still aren't telling".

Even given a court order, there's still a possibility that employing surveillance by fiat will cause somebody to leak, or modify how they handle data, or simply reveal information about what sort of surveillance tools a given agency employs. Given that a FISA order can be obtained without a defendant, getting a court order and then doing the thing secretly anyway gives a sort of "bowling with bumpers" advantage where the project is approved if it gets revealed, but also done without revealing anything if it isn't.

More disturbingly, there's also substantial evidence that the NSA attacks companies covertly in places where they couldn't get a court order. Taking a specific device out of the supply chain and adding surveillance before it's shipped to the destination is a warrant-worthy project. Setting up systematic physical vulnerabilities with a use case of "turn it on some time in the future to get something interesting" isn't in the purview of a FISA order, so if the NSA did do that it would have to be without an order.


Wasn't PRISM all about attacking American companies with covert implants? For instance tapping into Google region to region data transfers, after which Google started encrypting everything.


I thought PRISM wasn't covert. Companies were compelled to allow them to install their sniffing hardware, it was all above-board. Snowden even leaked an internal slideshow with a nice timeline of when each tech company joined the program.


Parts of that whole expose were covert. In the case of Google we know by tapping fiber connections that they had between data centers (as sseth mentioned, using foreign intelligence peers to do an end run around legal protections), which was on Google owned fiber, theoretically entirely "in-house", so Google transferred it unencrypted. I believe they called this operation "Muscular". After the fiasco Google started assuming everything was hostile.


PRISM was an overt program (to the data stewards, not to the public) for processing FISA warrants and the like.

Other NSA programs that Snowden revealed were covert hacks. https://en.wikipedia.org/wiki/Global_surveillance_disclosure...


My understanding is that the google tapping was done in UK using British intelligence services, thus bypassing the legal constraints.


The program for tapping data center links had the internal code name MUSCULAR and was a partnership with the British GCHQ, who actually did the intercepting.

PRISM was at first reported as some sort of direct access to the servers of certain American companies, but it turned out to be the code name for a joint program with the FBI for using FISA warrants to request data from those companies.


I used to work in engineering at one of the big wireless telecoms. The impression that I got was that many of the outsourced services were compromised. For instance, we had zero control over our voice mail systems, they were outsourced to Amdocs.

You can see how this benefits the NSA; if the voice mail is outsourced to a foreign company, and the NSA buys intel from that company, it's technically not spying on US citizens, particularly if they're getting metadata.


You don't know that. We do know that the USG covertly intercepted fiber communications.

https://www.washingtonpost.com/news/the-switch/wp/2013/11/04...


The story literally quotes the general of the NSA, saying they go though the FBI to get a FISA court order to compel the company..

Additionally, the story quoted talks about how the UK obtained the data and gave it to the NSA.

Nowhere is the NSA installing covert implants. They just don't do that.

The CIA does that :)


> The story literally quotes the general of the NSA

Ah, so he pinky-promised? Well OK then!


What I meant was that you don’t know it isn’t done.

You are taking the word of a spy? Did he say it wittingly?


Sure! It was the least untruthful thing he could say.



"SSL added and removed here ;-)" doesn't sound like a FISA court order.


The NSA has "attacked" internet infrastructure for many years before it became sort of legal (but probably still unconstitutional).


but the NSA has been performing backdoors on hardware for years

PRISM

https://www.schneier.com/blog/archives/2018/08/backdoors_in_...

> Juniper has confirmed that an initial analysis of malware linked to the National Security Agency appears to affect its firewalls.

https://www.zdnet.com/article/juniper-confirms-leaked-nsa-ex...


China is just as interested in US' communications.

Huawei & ZTE have been alleged with these exact type of attacks, by the US government


> the NSA would make sense as the source of the implant.

That doesn't make sense based on the assumption that US telecom companies already cooperate extensively with US inteligence agencies.


"Extensively" is not 100%


The only US Telecom that did not allow NSA direct access to vacuum up transmissions was Qwest, and their CEO was sent to prison.


..for insider trading. You are implying that he went to prison because of the NSA. He went to prison because he sold $52 million in stock after the intelligence community said they would no longer consider Qwest for classified government contracts because of his refusal to cooperate with the NSA.

He went to prison because he sold stock based on insider information. Regardless of the reasons for his trade, it was still insider information.


Everyone breaks laws all the time. The question is whether the Government decides to focus on your activities in order to identify your crimes.


I find these attempts to distinguish between different state sponsored criminals to be a diversion and subterfuge.

Whether China or the US re-allocates your IP, you can expect a competing product made in China. That you might have a relationship with one of them probably doesn't change anything unless they actually think your firm is the best one for the job of maximizing the results on their tax base.

I mean maybe I'm wrong; would it make any sense that these Republics take private corporate property more seriously other parts of their Constitutions they have violated at least until caught?

The US makes the claim China does not and former president and CIA head George Bush floated corporate espionage as THE plan for handling the absurd costs of "intelligence" criminals after the cold war..

I'm always astounded that working in a competitive market seems to blind people to significant stated facts of the environment their market is operating in.

In nature, it might make sense to just outrun the weakest, after all, a bear has a limited appetite. But superpowers have unlimited apetites and will collapse like the USSR if they should ever expand slower than cancer.


Is it illegal to report an intelligence attack that is perceived to be foreign? If not, why not have all attack reports assume they are foreign to begin with? This would give the reporter credible deniability, and put the burden on the US government to argue otherwise. Regardless, the report is released without the reporter getting in hot water. Or am I missing something?


Incentive for a company to say "No" when the FBI offers to "fix" the problem quietly either by going up the chain of command internally to get answers and stop a blown attack on a US owned and operated business or use contacts within the US security infrastructure to stop the foreign criminal or state adversary.

Most of these attacks never leave the room at corporate HQ where they are discovered unless an engineer wants to permanently screw themselves out of a career.

I once tried to leave a linkedin recommendation for a friend I'd worked with on a high profile project where he discovered Chinese state actors performing corporate espionage and we stopped it. The FBI came in and carted off the servers, we switched data centers, re-deployed, and that was that. We would never have been the wiser if he weren't closely monitoring network characteristics. 3 years and 2 job changes later he messaged me back to say, "Thank you for the rec. but don't mention that shit!"


>it's illegal to report an attack by US intelligence agencies

Could you expand more on that?


IANAL, but New York Times Co. v. United States is a famous precedent for the first amendment protecting the press' right to publish classified government documents.


That's what I was kind of getting at, the press is free to publish pretty much anything they'd like AFAIK.


> because it's illegal to report an attack by US intelligence agencies

is it? and if you were witness to such an attack, how would you be able to attribute it to the USA vs some other actor?


> it's illegal to report an attack by US intelligence agencies

Is this true? I mean outside a specific gag order or working under a clearance, you could find an issue with a piece of equipment, publicly talk about it and then be arrested because it turned out it was the US government who caused the issue?


> The "trojan ethernet connector" paragraph mentions similarity to an NSA implant, which appears to be this: https://en.wikipedia.org/wiki/NSA_ANT_catalog#/media/File:NS....

Interesting. This one seems to rely on the presence of the USB connectors for powering. So, basically, any ethernet port with only Ethernet connectors should be safe from this kind of attack? The only powering option there should be PoE but I have yet to see someone buy PoE switches for a datacenter...


"any ethernet port with only Ethernet connectors should be safe from this kind of attack? "

I know that some IPMI implementations can control/use the on board ethernet. There might be some hole there. Similar for the built in management on some Intel processors and nics.


>but I have yet to see someone buy PoE switches for a datacenter

It was a small datacenter but we had a bunch for PoE KVMs.


IANASecurityExpert but one one the named sources of the original claim also came out to say that the 'original attack' was a conceptual idea he had but that doesn't even make sense to apply in practice (considering better alternatives) http://appleinsider.com/articles/18/10/08/security-researche...


> it's illegal to report an attack by US intelligence agencies

Can you provide a citation for this statement please


[flagged]


>> I'm now wondering

> You can't use uncertainty [...] as evidence for your own nuttery. Stop it.

To wonder aloud should not be so demonized much less demanded to cease.


If you diagram out that sentence, you'll find it wasn't the wondering I was "demanding to cease".

I mean, sure, it could be NSA. Let's wonder! The fact that no one in public has seen one of those things isn't supporting evidence for that theory. It's even less grounded than the Chinese one.


Who knows... but it is funny that healthy skepticism can transform into this kind of acceptance. First disbelief, then, well ok, maybe but if it is, it’s probably not the suspected entity, but an altogether different entity... it’s like multiplying probabilities but thinking it increases likelihood.


Conspiracy theories do love the combination of the whataboutism and the competitive debate strategy of "spreading" where you make lots of weak (or better yet, completely unsubstantiated) points, and if your opponent fails to refute everyone, you win.


~


Just about anyone on Google's Project Zero team, off the top of my head. They would probably be both competent and enthusiastic.

Beyond that Apple, Microsoft, Tesla and probably Amazon have sufficiently capable public (or past public) researchers working for them. But I doubt many would want the publicity one way or the other.


Most any security company would LOVE to get access to the purported hacks and study them.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: