Hidden Gems from the Embedded Online Conference Archives - Part 2

• Software Development
• Project Management
• System Design

In the years since its inception, the Embedded Online Conference has hosted hundreds of excellent talks. The topics cover an impressive range, from high level introductions, to tool and framework comparisons, to case studies of really annoying bugs! This is the second post in a series looking at a different talk from the archives each week, appraising the content and providing some extra context. Here is the previous installment: Part 1.

This week's talk was a keynote in 2021, the first EOC I attended: Mars Perseverance Software by Steve Scandore (and Q&A).

Steve has been working at NASA's Jet Propulsion Laboratory (JPL) for nearly 30 years, with well over a decade focussed on flight software for missions to Mars. In 2021, as well as giving this fascinating talk, he was awarded the JPL Principal Designation which recognises "outstanding individual contributions in advancing scientific or technical knowledge". He continues to work on missions to the "red planet" such as Mars Sample Return.

In this talk, Steve gives insight into the history of Mars rovers, how the flight software works, and the challenges and constraints faced on the Perseverance project. Futhermore, Steve answered questions in a Q&A which was longer than his talk and is packed with interesting information! One of the benefits of the EOC format is that attendees can watch the talks in their own time, do some more research and bring specific questions to the Q&A. I am sure I am not alone when I say that stories of space exploration inspired me to become an engineer. Steve effectively breaks down the flight software behind these stories is created, giving lessons we can all apply to our own work, whether on Earth or not!

Beginning with an overview of the Mars 2020 mission, Steve introduces the Mars Rover Family, which dates back to 1997 with the Sojourner rover weighing in at 11.5 kg and operating for 95 days. The capability of these rovers has increased over time along with their size. This led to the Perseverance Rover which weighs 1,025 kg and at time of writing has driven over 30 km and is is still performing measurements over 4 years after landing. Oh, and it also had a helicopter drone, Ingenuity, which performed the first flights on another planet. These previous missions had a big impact on Steve's work, which built heavily on established software and hardware, in particular that of the Curiosity rover. In many ways, the Perseverance flight software is legacy software and the platform inherited many of the hardware restrictions such as memory size and processing power. This was done to speed up the development time and avoid some of the risks associated with new systems. Despite this, Steve says,

"People don't want you to remove any of the capabilities from the last mission. They want you to add capabilities."

This sentiment will be familiar to many engineers! Steve highlights the importance of the software architecture, which "allowed us to rework large parts of the system, without worrying about how that might impact other parts of the system". This architecture has elements which can be traced back to the Sojourner rover.

During the main section of the talk, Steve discusses the flight software in detail and from multiple angles. This software runs on the specially designed Rover Compute Elements (RCE) and is responsible for flying to Mars, Entry, Descent and Landing (EDL) and then roving around on Mars doing scientific experiments. At this point, I have to ask the reader to review the "Sky Crane" landing method used by Perseverance and the footage taken on board during the landing. Sky crane involves a separate descent stage which uses rockets to stabilise and then gently lower down the rover on ropes. I often have fun thinking about how people would reacted to this idea in meetings before deciding to go for it (one retelling here). So the flight software manages each of these stages and reduces the need to communicate back to Earth during operation, compared to previous missions.

There are many different ways to diagram an embedded system, and Steve covers quite a few during the talk. The hardware is explained along with the interfaces between different modules, some of which double as data paths. The software is also shown from a functional perspective, separating out the different layers. Steve also digs into the details of some specific functions, such as how a database of rocks allows the rover to know where it is and make autonomous decisions. By showing these different views of the system, Steve reveals how complex it is. As he says, there are enough challenges to cover several presentations! The software ended up with around 1.2 million lines of code. And in the end, "it all worked". How did Steve and the team manage this?

Well, they started with a lot of tests. There is more test code than flight code, giving 100% code coverage. They designed an avionics simulation package called the Work Station TestSet (WSTS) which all developers could use. A second rover, Optimism, was built to perform tests at full scale on Earth. And they obeyed the JPL Coding Standard. Steve says that although they did not strictly follow test-driven development (TDD), they do require regression tests to be added when issues are found. This approach - broadly containing conding standards, unit tests, integration tests, simulation and hardware in the loop tests - led to a very reliable complex system developed by a large team.

Approaching the end of the talk, Steve gives positive messages about the impact of the Perseverance mission and engineering more widely which I will leave you with:

We in this industry, us, the aerospace industry, the embedded computing industry, what we do has a positive impact on humanity.

I hope you've enjoyed this look into the Embedded Online Conference archives. Join me again next week for another "hidden gem"!

All of these EOC talks were referenced by Steve during his talk on Mars Perseverance Software:

• Modern Embedded Software Goes Beyond the RTOS by Miro Samek
• Best Practices for Developing Real-time Embedded Systems by Jacob Beningo
• Operating Systems for Embedded Applications by Colin Walls

Further Reading:

• Cost-Cutting Flight Infrastructure Improvements on M2020 by JPL
• How we land on Mars by Nasa

Sign in