New telemetry management company Sift, fresh from a USD $7.5 million fundraising round and an exit from stealth, is looking to leverage their founders’ SpaceX experiences to help manage and store the vast quantities of data generated by space companies, and by many other hardware-focused companies with large quantities of machine-generated data.
While not necessarily a household name, telemetry is a key part of the space industry. It’s the data that sensors in a space-based vehicle, satellite, or other device gathers about their situation and conditions. Sensors can detect everything from temperature, acceleration, and position in IoT devices to critical life support information for crewed spacecraft, and the information they send back can be both critical and immense.
Dealing with telemetry and using it effectively is a key task for both hardware and software engineers at space-focused companies. It’s also not an easy one. For Sift founders (and former SpaceX engineers) Karthik Gollapudi and Austin Spiegel, this kind of software was not only key to their contributions at SpaceX’s efforts, but also the reason why they decided to form their own company.
In an interview with SpaceQ involving Gollapudi, Spiegel, and Sift Head of Marketing Nat Brown-Lennox, they gave more detail on what their company is about.
Software Failure and Starliner
Gollapudi was Flight Software Lead for SpaceX’s Crew Dragon, and that job requires that you understand the paramount importance of software when it comes to safe crewed spaceflight.
With is experience, and when the Boeing’s OFT-1 demo Starliner mission failed, he assumed that it was a hardware problem of some kind. “I was used to this world where you see all these failures in hardware, like seeing the space shuttle and the frozen O-ring,” he said, “so I was really surprised when the press release came out [saying that it was] due to poor software testing with this cascading series of software failures.” While software bugs are a familiar part of the tech sector, he was still shocked that something as vital and expensive as the Starliner demo could have been thrown off by software issues.
It seemed alien to Gollapudi, because SpaceX places a big emphasis on reliable software, and particularly on the software focused on telemetry data used in R&D. But the more he looked around, the more he saw companies that had insufficient or unreliable software—a mess of jury-rigged in-house solutions that weren’t nearly reliable or scaleable enough for the job.
Spiegel had learned that lesson as well after leaving his job at SpaceX, as a Lead Software Engineer for the Starlink Constellation Team, back in 2020. He spent a fair amount of those two years talking with a host of companies that were having software issues related to telemetry. So when the two of them reconnected in 2022 and compared notes, they realized that there was a real market opportunity, so Karthik bid farewell to SpaceX and started building Sift with Spiegel.
After some time in stealth, they emerged earlier this year, offering software solutions for telemetry data and for hardware-generated data in general.
Sifting Through Immense Data
The Sift stack, basically, is a platform that ingests, transforms, stores, and analyzes a company’s data, with a focus specifically on telemetry. Spiegal called it “a verticalized data solution,” meaning that it handles all aspects of the process once the data has been generated and transferred to Sift.
Sift will allow you to perform real-time analysis on the data, finding problems and working out solutions, as well as setting expression-based rules and notifications to flag situations where (say) a particular sensor is providing data that may require immediate action or attention. So (as an example) while doing early R&D, you can discover a particular rare failure state, analyze it, and discern the stages that can lead up to that failure; then you can either fix it, or tell Sift to find early telemetry signs in operational hardware that it can use to alert and warn you of potential issues.
While this may seem straightforward and ubiquitous, it’s not. Spiegel pointed to their experience in space, saying that telemetry in space, particularly in R&D, has you “sampling that sensor somewhere up to 50,000 times a second,” leading to enormous quantities of data that need to be tracked at the tiniest increments possible, and often analyzed in real time. You also often have to be able to effectively compare wildly varying datasets; Spiegel said that “if you have a satellite in orbit, you’re continuously capturing data,” but the testing data you did on the ground “all happens for discrete periods of time.”
For these and other reasons, you can end up with datasets that are extremely difficult to integrate and compare, let alone analyze; which means that the signs you might have relied on in R&D won’t work the same when you go operational. Sift’s tools are intended to make that easier, even straightforward.
Gollapudi said that, in a lot of companies, the data can actually end up as gigantic multi-gigabyte spreadsheet files, hosted on individual users’ PCs. Then people at the company start emailing them to each other, and soon enough they end up needing to “figure out who emailed this around the company; people have different versions, they get lost, you can’t share them.”
“It’s a pain,” he added. Especially when a company’s leaders decide to start scaling up, and start having serious problems as they discover that a set of big spreadsheets just simply won’t do the job anymore.
So, Spiegel said, a big part of the job was figuring out that while “everyone wants to visualize telemetry and analyze it,” the formats were often either incompatible or, like the CSV files Gollapudi mentioned, simply unsuited to the job. They not only had to develop special APIs (computer-to-computer interfaces) for customers to use to standardize and transmit their data, but build a flexible schema within Sift to handle different data types.
Spiegel said that they already support 10 different types, though they’ve figured out that in many cases, the workflows are similar enough to make the job somewhat easier, and that they can work with clients to deal with the edge situations. They’re also working hard to create migration tools from legacy data stores and legacy systems to the Sift stack.
To deal with the immense quantities of data, they also use a multiple-tiered approach to storage. Like many companies, they rely on cloud storage. But because much of the data is generally written once and then only accessed when infrequently needed, Spiegel said that Sift relies a lot on object storage—cloud storage that’s organized into objects instead of files, folders, blocks or databases—which is generally seen as most appropriate for the massive amounts of write-once data that telemetry generates.
It’s often used in media, and in a variety of other cloud storage applications: IBM highlighted that “[t]oday’s Internet communications data is largely unstructured…[such as] email, videos, photos, web pages, audio files, sensor data, and other types of media and web content.” It’s also very suitable for analysis, as it’s kept in a flat structure using robust metadata. But a small company, or even hardware-focused engineers, may not even know where to start, or realize that it’s needed at all, even as they’re filling up hard drives with terabytes of sensor telemetry.
Not all of their customers’ data is stored this way, but they’ve built a system that can smoothly move it in and out as needed, saving on data storage costs while letting customers focus on other things and scale up their data storage as needed. And it contributes to Sift’s ability to generate robust visualizations of the data that customers can use as they’re developing their products.
Gollapudi said that this allows for a smooth flow from R&D to scaling and operations. While doing R&D he said, “you need better visibility into your machine, and what you want is a good way to manage this data as it’s coming off your test devices, so you can collaborate on building a new machine.” You’ll be generating gigantic amounts of data very quickly, from a number of different test stands, and need to analyze all of them just as quickly as you iterate on your new product.
Once companies move out of R&D and into scaling and operations, they’ll need to automate quality control and software updates. Gollapudi said that on quality control, users can create these automated expressions with Sift, making quality checks automated and unified across all of their operational devices. And in operations, he highlighted how they built Sift to be able to seamlessly run software update tests and analysis across a number of different platforms—including production equipment, test equipment, and fully computer-based simulations—helping to ensure that software updates are smooth, uniform, and bug-free.
Gollapudi said that, to a great extent, what they’re trying to bring from SpaceX is an attitude focused on “servicing risks early.” They want their tools to give companies the ability to use as much instrumentation and telemetry as possible, as early as possible, generating as much data as possible, so as to find and resolve problems as quickly as possible. “What we do,” he said, is “give you the tools to see ‘is this thing working or not’ while you’re still in the R&D phase,” rather than finding out after it’s far too late.
Exiting Space
Though the company’s genesis came from space, Gollapudi said that they’re setting their sights beyond the space industry. Their solutions seem to be roughly comparable, though he’s learned that “there’s different emphases on what’s important to them. Aviation is focused on certification,” he said, and needs reliable long-term data retention that will be available for submission to regulatory bodies like the FAA. “Whereas in space there’s a lot more focus on not blowing up…the cost of failure is really high.”
They’re venturing well outside of aerospace, however; Gollapudi said they have clients across a variety of sectors, including autonomous trading, nuclear energy, and the automotive sector. As they’ve expanded their reach, he said that they noticed that “the problems are relatively similar…it’s more [about] communicating which pain points we’re going after.”
The regulatory needs of the aviation industry, for example, appears to have led them to develop automated certification reports that draw on the Sift data stores. Regulatory compliance is a routine headache at many companies; any automation of that process may well be worthwhile for them in-and-of itself.
Regarding their recent $7.5 million fundraising round led by Riot Ventures with Fika Ventures, First Resonance, Datum and Duro however, they didn’t have much to add to what was already available in the recent release. Nat Brown-Lennox, head of Marketing at Sift, said that they are “solely focused on building the best product for our customers rather than going out and doing another raise.”
Instead, their scaling-up process is focused on getting talent, rather than funding; they have twelve people now, and she said that they are looking to double that “in the next six months.”