There's a fascinating and redacted interview with an "anonymous" subject about the disaster. To say the least it's an unsuccessful attempt to hide the identity of the individual:
"Q. So how did you get yourself started into submersible operations?
A. Well, I'm sure you're familiar with my film Titanic. When I set down the path to make that film, the first thing that I did was arrange to be introduced to the head of the submersible program at the P.P. Shirshov Institute in Moscow, a guy named..."
It appears that all the engineers -- system designer, material
engineer and structural analyst -- thought that OceanGate CEO was going to
kill himself:
If you ever find <name-of-the-engineer>, he’s not going
to have a whole lot of nice to say. He was very frustrated
with the company. (...) And I understand why. He thought
Stockton was going to kill himself.
And the director himself declined to dive on Titan when asked:
Now, the question is, why wouldn’t the engineer get inside
his own vehicle? It was because of what I felt -- and I have a
background in Navy diving in EOD operations. I knew firsthand
that the operations group was not the right group for that role,
and I told him as much, that I don’t trust operations and who he
has there.
The number of stupid decisions that went in the design and construction of the Titan is astonishing. One of my favorites was that, after putting on the carbon fiber around the tube, they would sand imperfections to make the surface perfectly smooth, severing layers in the process! It shouldn't require an engineering degree from MIT to recognize this as ill-advised.
Even without that, the material is just wrong. It’s strong in tension, not so much compression. Tends towards sudden brittle fractures. Doesn’t like impacts, as it tends to have issues with delaminating.
It’s just not what you ever want as a sub hull. It’s dumb.
Yes, using carbon fiber was also a very bad decision; it was known for a very long time that it was only good for single-use sub, because after the first dive it was too damaged to continue. In 2014, Virgin Oceanic, which had similar plans with similar technology, closed shop because it didn't make economic sense to build a new sub for each dive.
But weight is absolutely an issue; the basic and tried-and-true metal sphere design allows for only three people. Since size and thickness grow exponentially, making a sphere for more than three people becomes more and more difficult. And it should also be possible to lift the vehicle with a crane.
But if you want to carry paying passengers (like Oceangate did), having only two per dive is very limiting. That's why they went with a tube design, and carbon fiber to limit weight. But it couldn't work, and it didn't.
Ok yes "exponentially" was hyperbolic. Mass scales linearly with volume, but volume is proportional to the cube of the radius (not linear).
Also, in practice, small imperfections can have a disproportionate impact on the resistance of the sphere so design codes typically apply conservative reductions that can have a big impact on actual thickness requirements.
I read that the pilot was also basically suicidal. His wife had died, and he was completely fine with the danger because he would die doing what he loved, and he didn't really want to live anymore.
> Wreck expert Paul-Henri "P.H." Nargeolet, who was also onboard, told me he wasn't worried about what would happen if the structure of the Titan itself were damaged when at the bottom of the ocean. "Under that pressure, you'd be dead before you knew there was a problem." He said it with a smile.
(as recounted by Arnie Weissmann, in Travel Weekly article published June 22, 2023)
There was only one other crew member in that vessel (well, actual crew and not paper "mission specialist"). He was an older gentleman, and it's quite common for older people to have lost their spouse. Was that so hard to figure out?
Took me forever to find the actual quote - ChatGPT and Gemini kept trying to gaslight me that it’s not a real quote or that Willie said it to Bart until I gave the exact quote (at which point normal search engines were fine):
> A certain agitator, for privacy's sake, let's call her Lisa S. No, that's too obvious, let's say L. Simpson.
The SD card on the camera was intact but encrypted. Decrypting the data required a key stored on a separate SOM board, but the SOM was damaged. The investigation team delivered the SOM and SD card to the camera manufacturer in Newfoundland, and they were able to decrypt the card.
They found a couple of images, but
No data with a timestamp after May 16th was found on the camera, so it is likely that none of the data recorded on the SD Card were of the accident voyage or dive.
After all that work...
If you're interested in data recovery, you will enjoy reading this report, about 10 pages, clearly written. The technical language mentioned they didn't see a LUKS header on the card so they figured it was a custom dm_crypt setup.
> No data with a timestamp after May 16th was found on the camera, so it is likely that none of the data recorded on the SD Card were of the accident voyage or dive.
Evidently the camera data was recorded to an external SSD card
in the mission computer when the accident occurred.
The investigation team actually managed to salvage the PC as well:
> To conduct the CT scans, the large mass was evaluated by a third-party
laboratory under NTSB supervision. This facility had a range of scanners with different
power and energy levels and could scan large masses using a rotating table, avoiding
the need to rotate the mass itself. Ultimately, the third-party laboratory attempted to
image the large mass at a power as high as 320 kilovolts (kV). The scans conducted at
320 kV were not powerful enough to penetrate the object, and as a result, no internal
structures or voids were visible, and no memory devices could be identified. The
NTSB evaluated using another laboratory with higher power and energy CT scan
devices, however, there was concern that increased CT scan energy could damage
data stored on any surviving NVM chips. Consequently, higher-energy scans were not
pursued.
I'm no expert, but remember reading about neutron imaging ([1]). I'm curious if that was deemed unfeasible, too expensive, or having little chance of success? From Wikipedia:
> X-rays are attenuated based on a material's density. Denser materials will stop more X-rays. With neutrons, a material's likelihood of attenuation of neutrons is not related to its density. Some light materials such as boron will absorb neutrons while hydrogen will generally scatter neutrons, and many commonly used metals allow most neutrons to pass through them.
Wow. SubC’s software engineering needs some work. They thought the camera’s file system was unencrypted, when it was encrypted. They didn’t know where the keys were to decrypt it. It turned out the key was written unencrypted to a UFS storage device. There was a file written to /mnt/nas/Stills, which indicates that the camera was to writing to a remote file system that wasn’t mounted.
They thought the camera’s file system was unencrypted, when it was encrypted.
Unfortunately this situation is likely to get more common in the future as the "security" crowd keep pushing for encryption-by-default with no regard to whether the user wants or is even aware of it.
Encryption is always a tradeoff; it trades the possibility of unauthorised access with the possibility of even the owner losing access permanently. IMHO this tradeoff needs careful consideration and not blind application.
This is why I always shake my head when the Reddit armchair security experts say "The data wasn't even encrypted!? Amateur hour!" in response to some PII leak.
Sure, sure buddy, I'll encrypt all of my PII data so nobody can access it... including the web application server.
Okay, fine, I'll decrypt it on the fly with a key in some API server... now the web server had unencrypted access to it, which sounds bad, but that's literally the only way that it can process and serve the data to users in a meaningful way! Now if someone hacks the web app server -- the common scenario -- then the attacker has unencrypted access!
I can encrypt the database, but at what layer? Storage? Cloud storage is already encrypted! Backups? Yeah, sure, but then what happens in a disaster? Who's got the keys? Are they contactable at 3am?
Etc, etc...
It's not only not as simple as ticking an "encrypted: yes" checkbox, it's maximally difficult, with a very direct tradeoff between accessibility and protection. The sole purpose of encrypting data is to prevent access!
Server stores encrypted blobs. Server doesn't have the keys.
Entire application is on the client, and just downloads and decrypts what it needs.
Obviously your entire application stack needs to be developed with that approach in mind, and some things like 'make a hyperlink to share this' get much more complex.
> They thought the camera’s file system was unencrypted, when it was encrypted.
Willing to bet plenty of hn readers are unaware of encryption going on at lower layers of the tech stack than they're aware of.
For example most hard drives encrypt all data, even when not commanded to, as a way to do 'data whitening' (ie making sure there are even numbers of 0's and 1's in the data stream and not some pattern which might throw off tracking.)
The encryption key is simply stored elsewhere in the drive - or nvram or in the firmware.
But it means if you extract the physical magnetic surface and read it with the right microscope, you might well find the data encrypted with no available key.
If you're talking about SED feature, no, it isn't widespread since it's regarded as an "enterprise" feature and only available in minority of drives (regardless of HDD or SSD).
Client or OEM variants of same drives (otherwise identical) lack SED option most of the time and doesn't encrypt data by default.
Scrambling and encryption are two different things. Scrambling is very easy to do at line rates. Encryption not so much.
Ethernet is a good example. It has the same problem where long strings of 0's or 1's can cause clock recovery problems. The solution as clock rates have increased is to just run all the data through a scrambler driven by a simple Linear Feedback Shifter.
This is line coding, often used on wired connections. But reading a hard drive trace isn't quite a wired connection, so the trade-offs are different.
Most notably with line coding when using positive and negative voltages it is quite important for the average voltage to be zero to avoid building up a charge difference.
Whitening can often be used if the downside to an imbalance or long runs is much lower. Notably in RF this is often about avoid harmonics with a little bit of symbol timing advantage thrown in.
Whitening doesn't really require encryption though. Weak cypher streams xored into the data work fine. Even a repeated 256 bit string is quite alright.
Whitening using any non trivial encryption key seems weird to me. AES with a key equal to the current offset in ECB mode already feels over-engineerd.
> Whitening using any non trivial encryption key seems weird to me.
It's because there was an era when drives were expected to be able to do 'hardware' encryption with a user provided key, so reusing that hardware to also do whitening even if the user didn't provide a key was very convenient.
Plus you get all the other benefits - ie. a single scsi command can 'secure erase' the whole disk in milliseconds by simply changing the stored key.
> Removed SD card. The manufacturer of the camera had requested certain components of
the device be redacted. Portions of this image have been redacted.
And so it is, but anyone who has ever seen a Sandisk SD card knows what they're looking at. I can even tell it's not the fastest V90 speed.
The things companies try ineffectually to keep out of public view are weird.
Amusing that the bits the “manufacturer asked to be redacted” in the images appear to be the identifiers for common off-the-shelf electronic components, including a standard memory card. Is that really super secret IP?
I'm confused. Why are decryption keys in NVRAM? That seems to negate the purpose of at-rest encryption if you can retrieve keys from the device even after shutdown.
Well they're encrypting an SD card, so this still defends against its being removed from the camera and stolen or left in a bar or something.
But honestly from the rest of the story it sounds like the camera manufacturer was selling their pressure housing moreso than the off-the-shelf camera hardware inside, and was not particularly concerned with whether/how the storage was encrypted.
Crazy that it's pretty much a 3D printed assembly internally, and the manufacturer didn't know how it worked. No way that would pass any kind of vibration test.
The "carrier" that everything rides on within the housing is clearly FDM printed as well. I assume these cameras (rated to 6,000 meters) are rather low volume products.
It honestly makes sense. You are paying for the pressure engineering, and can take advantage of an off the shelf camera system. Maybe use a special lens or filter or something but why bother customizing the software/hardware of the camera much.
They probably should still know what it's doing though...
I have seen engineers slap Teensies on a PCB and call it a day, so it’s definitely normal. It’s faster than having to route your MCU, USB, debugger, etc. manually, so there isn’t really a drawback as long as it physically fits there.
Common misconception. A handful of capacitors, SPI NOR flash, an inductor, and a crystal is way easier to place and route than a restrictive module that completely disables your ability to use SWD/JTAG on an otherwise excellent MCU.
There's a fascinating and redacted interview with an "anonymous" subject about the disaster. To say the least it's an unsuccessful attempt to hide the identity of the individual:
"Q. So how did you get yourself started into submersible operations?
A. Well, I'm sure you're familiar with my film Titanic. When I set down the path to make that film, the first thing that I did was arrange to be introduced to the head of the submersible program at the P.P. Shirshov Institute in Moscow, a guy named..."
https://media.defense.gov/2025/Sep/17/2003800984/-1/-1/0/CG-...
Among the interviews, one with the former engineering director was the most eye-opening for me.
https://data.ntsb.gov/Docket/Document/docBLOB?ID=17236880&Fi...
It appears that all the engineers -- system designer, material engineer and structural analyst -- thought that OceanGate CEO was going to kill himself:
And the director himself declined to dive on Titan when asked:The number of stupid decisions that went in the design and construction of the Titan is astonishing. One of my favorites was that, after putting on the carbon fiber around the tube, they would sand imperfections to make the surface perfectly smooth, severing layers in the process! It shouldn't require an engineering degree from MIT to recognize this as ill-advised.
Even without that, the material is just wrong. It’s strong in tension, not so much compression. Tends towards sudden brittle fractures. Doesn’t like impacts, as it tends to have issues with delaminating.
It’s just not what you ever want as a sub hull. It’s dumb.
And weight is not even a huge issue for a sub!
Yes, using carbon fiber was also a very bad decision; it was known for a very long time that it was only good for single-use sub, because after the first dive it was too damaged to continue. In 2014, Virgin Oceanic, which had similar plans with similar technology, closed shop because it didn't make economic sense to build a new sub for each dive.
But weight is absolutely an issue; the basic and tried-and-true metal sphere design allows for only three people. Since size and thickness grow exponentially, making a sphere for more than three people becomes more and more difficult. And it should also be possible to lift the vehicle with a crane.
But if you want to carry paying passengers (like Oceangate did), having only two per dive is very limiting. That's why they went with a tube design, and carbon fiber to limit weight. But it couldn't work, and it didn't.
Ok yes "exponentially" was hyperbolic. Mass scales linearly with volume, but volume is proportional to the cube of the radius (not linear).
Also, in practice, small imperfections can have a disproportionate impact on the resistance of the sphere so design codes typically apply conservative reductions that can have a big impact on actual thickness requirements.
It’s funny how “literally” often means “figuratively” now, and “exponentially” means “polynomially”.
Yep.
https://books.google.com/ngrams/graph?content=exponentially%...
I read that the pilot was also basically suicidal. His wife had died, and he was completely fine with the danger because he would die doing what he loved, and he didn't really want to live anymore.
Wasn't the pilot Stockton Rush? His wife was alive. Who are you referring to? I tried to check your claim but I couldn't verify it.
they're talking about nargeolet
> Wreck expert Paul-Henri "P.H." Nargeolet, who was also onboard, told me he wasn't worried about what would happen if the structure of the Titan itself were damaged when at the bottom of the ocean. "Under that pressure, you'd be dead before you knew there was a problem." He said it with a smile.
(as recounted by Arnie Weissmann, in Travel Weekly article published June 22, 2023)
Yeah I got the roles of pilot and guide mixed
https://www.newyorker.com/news/a-reporter-at-large/the-titan...
There was only one other crew member in that vessel (well, actual crew and not paper "mission specialist"). He was an older gentleman, and it's quite common for older people to have lost their spouse. Was that so hard to figure out?
yeah but he remarried
Let’s hear from L Simpson. No, that’s too specific. Let’s hear from Lisa S.
Took me forever to find the actual quote - ChatGPT and Gemini kept trying to gaslight me that it’s not a real quote or that Willie said it to Bart until I gave the exact quote (at which point normal search engines were fine):
> A certain agitator, for privacy's sake, let's call her Lisa S. No, that's too obvious, let's say L. Simpson.
Lisa the Vegetarian
If you google “Lisa S L Simpson” it’s the first thing that comes up… why make it harder with AI?
… Quite ironically, Google results are AI… :)
In the future, Frinkiac [0] is your friend
[0] https://frinkiac.com/
When I grow up I'm going to Bovine University!
Cameron did several TV interviews about the Titan, why would they redact his name?
He didn't mince his words either; he was extremely critical of the whole thing before and after the disaster.
"I think the most dangerous part of our whole operation was these young software engineers puking over the railing in a high sea."
The SD card on the camera was intact but encrypted. Decrypting the data required a key stored on a separate SOM board, but the SOM was damaged. The investigation team delivered the SOM and SD card to the camera manufacturer in Newfoundland, and they were able to decrypt the card.
They found a couple of images, but
After all that work...If you're interested in data recovery, you will enjoy reading this report, about 10 pages, clearly written. The technical language mentioned they didn't see a LUKS header on the card so they figured it was a custom dm_crypt setup.
> No data with a timestamp after May 16th was found on the camera, so it is likely that none of the data recorded on the SD Card were of the accident voyage or dive.
Evidently the camera data was recorded to an external SSD card in the mission computer when the accident occurred.
The investigation team actually managed to salvage the PC as well:
https://data.ntsb.gov/Docket/Document/docBLOB?ID=19169363&Fi...
Sadly it turned into a compressed ball of metal...
From the report:
> To conduct the CT scans, the large mass was evaluated by a third-party laboratory under NTSB supervision. This facility had a range of scanners with different power and energy levels and could scan large masses using a rotating table, avoiding the need to rotate the mass itself. Ultimately, the third-party laboratory attempted to image the large mass at a power as high as 320 kilovolts (kV). The scans conducted at 320 kV were not powerful enough to penetrate the object, and as a result, no internal structures or voids were visible, and no memory devices could be identified. The NTSB evaluated using another laboratory with higher power and energy CT scan devices, however, there was concern that increased CT scan energy could damage data stored on any surviving NVM chips. Consequently, higher-energy scans were not pursued.
I'm no expert, but remember reading about neutron imaging ([1]). I'm curious if that was deemed unfeasible, too expensive, or having little chance of success? From Wikipedia:
> X-rays are attenuated based on a material's density. Denser materials will stop more X-rays. With neutrons, a material's likelihood of attenuation of neutrons is not related to its density. Some light materials such as boron will absorb neutrons while hydrogen will generally scatter neutrons, and many commonly used metals allow most neutrons to pass through them.
[1] https://en.wikipedia.org/wiki/Neutron_imaging#Neutron_radiog...
That's a striking image! Thanks for sharing - that really hits home on the pressures involved.
Pretty sure tech exists to recover data from flash memory with cracked dies...
I guess they decided it wasn't worth pursuing.
> Pretty sure tech exists to recover data from flash memory with cracked dies...
If you have anymore on this would love to see any relevant materials.
You can just make out the heatsink fins of the three PCs there, stacked atop (and now kind of inside) each other.
That truly is one of those “let God sort them out” situations.
Previous discussion (October 17th): https://news.ycombinator.com/item?id=45613898
Also a good video from Scott Manley: https://youtu.be/qMUjCZ7MMWQ
Wow. SubC’s software engineering needs some work. They thought the camera’s file system was unencrypted, when it was encrypted. They didn’t know where the keys were to decrypt it. It turned out the key was written unencrypted to a UFS storage device. There was a file written to /mnt/nas/Stills, which indicates that the camera was to writing to a remote file system that wasn’t mounted.
They thought the camera’s file system was unencrypted, when it was encrypted.
Unfortunately this situation is likely to get more common in the future as the "security" crowd keep pushing for encryption-by-default with no regard to whether the user wants or is even aware of it.
Encryption is always a tradeoff; it trades the possibility of unauthorised access with the possibility of even the owner losing access permanently. IMHO this tradeoff needs careful consideration and not blind application.
This is why I always shake my head when the Reddit armchair security experts say "The data wasn't even encrypted!? Amateur hour!" in response to some PII leak.
Sure, sure buddy, I'll encrypt all of my PII data so nobody can access it... including the web application server.
Okay, fine, I'll decrypt it on the fly with a key in some API server... now the web server had unencrypted access to it, which sounds bad, but that's literally the only way that it can process and serve the data to users in a meaningful way! Now if someone hacks the web app server -- the common scenario -- then the attacker has unencrypted access!
I can encrypt the database, but at what layer? Storage? Cloud storage is already encrypted! Backups? Yeah, sure, but then what happens in a disaster? Who's got the keys? Are they contactable at 3am?
Etc, etc...
It's not only not as simple as ticking an "encrypted: yes" checkbox, it's maximally difficult, with a very direct tradeoff between accessibility and protection. The sole purpose of encrypting data is to prevent access!
I like the approach of mega.nz...
Server stores encrypted blobs. Server doesn't have the keys.
Obviously your entire application stack needs to be developed with that approach in mind, and some things like 'make a hyperlink to share this' get much more complex.Nah bro, you just gotta use homomorphic encryption! /s
That said, encryption at rest is still good in terms of theft or mis-disposal.
> They thought the camera’s file system was unencrypted, when it was encrypted.
Willing to bet plenty of hn readers are unaware of encryption going on at lower layers of the tech stack than they're aware of.
For example most hard drives encrypt all data, even when not commanded to, as a way to do 'data whitening' (ie making sure there are even numbers of 0's and 1's in the data stream and not some pattern which might throw off tracking.)
The encryption key is simply stored elsewhere in the drive - or nvram or in the firmware.
But it means if you extract the physical magnetic surface and read it with the right microscope, you might well find the data encrypted with no available key.
If you're talking about SED feature, no, it isn't widespread since it's regarded as an "enterprise" feature and only available in minority of drives (regardless of HDD or SSD). Client or OEM variants of same drives (otherwise identical) lack SED option most of the time and doesn't encrypt data by default.
The hardware is still there - they don't make a new asic for the consumer version
doesn't mean it's active.
Scrambling and encryption are two different things. Scrambling is very easy to do at line rates. Encryption not so much.
Ethernet is a good example. It has the same problem where long strings of 0's or 1's can cause clock recovery problems. The solution as clock rates have increased is to just run all the data through a scrambler driven by a simple Linear Feedback Shifter.
To my knowledge, encoding avoids runs to avoid desynchronization in a way that isn’t encrypted.
This is line coding, often used on wired connections. But reading a hard drive trace isn't quite a wired connection, so the trade-offs are different.
Most notably with line coding when using positive and negative voltages it is quite important for the average voltage to be zero to avoid building up a charge difference. Whitening can often be used if the downside to an imbalance or long runs is much lower. Notably in RF this is often about avoid harmonics with a little bit of symbol timing advantage thrown in.
Whitening doesn't really require encryption though. Weak cypher streams xored into the data work fine. Even a repeated 256 bit string is quite alright.
Whitening using any non trivial encryption key seems weird to me. AES with a key equal to the current offset in ECB mode already feels over-engineerd.
> Whitening using any non trivial encryption key seems weird to me.
It's because there was an era when drives were expected to be able to do 'hardware' encryption with a user provided key, so reusing that hardware to also do whitening even if the user didn't provide a key was very convenient.
Plus you get all the other benefits - ie. a single scsi command can 'secure erase' the whole disk in milliseconds by simply changing the stored key.
This used to be done, but since ~2000, disks were expected to support on-device encryption, and by making encryption always-on no encoding is needed.
> Removed SD card. The manufacturer of the camera had requested certain components of the device be redacted. Portions of this image have been redacted.
And so it is, but anyone who has ever seen a Sandisk SD card knows what they're looking at. I can even tell it's not the fastest V90 speed.
The things companies try ineffectually to keep out of public view are weird.
Report on unrecoverable SSDs:
https://data.ntsb.gov/Docket/Document/docBLOB?ID=19169363&Fi...
Full docket:
https://data.ntsb.gov/Docket/?NTSBNumber=DCA23FM036
Amusing that the bits the “manufacturer asked to be redacted” in the images appear to be the identifiers for common off-the-shelf electronic components, including a standard memory card. Is that really super secret IP?
I'm confused. Why are decryption keys in NVRAM? That seems to negate the purpose of at-rest encryption if you can retrieve keys from the device even after shutdown.
Well they're encrypting an SD card, so this still defends against its being removed from the camera and stolen or left in a bar or something.
But honestly from the rest of the story it sounds like the camera manufacturer was selling their pressure housing moreso than the off-the-shelf camera hardware inside, and was not particularly concerned with whether/how the storage was encrypted.
Crazy that it's pretty much a 3D printed assembly internally, and the manufacturer didn't know how it worked. No way that would pass any kind of vibration test.
What's with the entire dev board crammed in there? Is that... normal? What board is it?
It appears to be a Teensy 3.2
The "carrier" that everything rides on within the housing is clearly FDM printed as well. I assume these cameras (rated to 6,000 meters) are rather low volume products.
It honestly makes sense. You are paying for the pressure engineering, and can take advantage of an off the shelf camera system. Maybe use a special lens or filter or something but why bother customizing the software/hardware of the camera much.
They probably should still know what it's doing though...
I have seen engineers slap Teensies on a PCB and call it a day, so it’s definitely normal. It’s faster than having to route your MCU, USB, debugger, etc. manually, so there isn’t really a drawback as long as it physically fits there.
> It’s faster than having to route your MCU
Common misconception. A handful of capacitors, SPI NOR flash, an inductor, and a crystal is way easier to place and route than a restrictive module that completely disables your ability to use SWD/JTAG on an otherwise excellent MCU.
The small board on the left is unmistakably a Teensy 3.2:
https://www.pjrc.com/store/teensy32.html
As to what it's doing in there, I have no idea.
Well, I'll be darned! I wonder if Paul Stoffregen knows about that!
edit: probably? It was posted at the Teensy forum about a month ago.
https://forum.pjrc.com/index.php?threads/the-deepest-teensy....
Looks like a pi zero
[dead]