Making an Impact as a Scrum Master

"You don't want to work for the methodology, you want the methodology to work for you."

I recently joined Aurora Labs as scrum master and wanted to reflect on my first months at the company. My job involves managing the agile workflow, hosting daily meetings, coaching team members, and removing roadblocks. It's rewarding work; ensuring the smooth running of feature releases, bug fixes, updates, and more across the Aurora Labs platform.

Before joining the company, I worked as a front-end developer and technical scrum master, splitting my time between the two roles. After two years of this, I wanted to transition to a 100% scrum master role and step outside my comfort zone.

I looked for a growing startup that could benefit from my agile skills but would provide the kind of challenge I was looking for. When you work for a big company, everything is clear, and everything is defined, but I wanted the chance to bring more of myself and my ideas into a business that would give me the freedom to push myself and drive things forward.

Aurora Labs seemed like the perfect fit and I was eager to get started but I knew there would be a challenge in getting up to speed with the automotive industry and a new way of working. The first task was to get to know the business, its processes, and all the technologies it uses. A priority for me was to meet the people, too, as it's important to me to understand the culture of a business and where I fit in.

 

The differences

Working in a large company has challenges and is a very different environment than working in a startup with a close-knit team. This gave me the kind of freedom I needed to make a real impact in my work.

At Aurora Labs, you have the opportunity to pinpoint an idea or change, followed by a proof of concept. From there you bring it to the relevant people to have a discussion and explain what needs to be done. This means that changes can be made quickly and you can really see how your ideas improve and support the process. This is exactly the kind of environment I wanted to be in.

Another highlight was the buy-in from the Aurora Labs team. The CEO gave me a daily time slot in his calendar to talk about the things I needed in order to succeed. It was clear that it's really important for him and for the company to be more agile.

Additional support

Around the time I started, Aurora Labs also began work with external agile coach Boaz Fine to support me and others in the business. This improved how quickly I and fellow scrum master Gil Baruch were able to get up to speed. We worked alongside Fine to brainstorm the next steps, then discuss the right way to ensure smooth changes. This was the agile team that drove the impact on the organization forward.

 

Making an impact

When I joined Aurora Labs, the company was using the Spotify agile model. It had been working well with a clear focus on scaling agile processes while emphasizing the importance of culture and network. This is just one example of structuring teams and sprints in order to get work done in an efficient way.

It soon became apparent, however, that this wasn't quite the right model for the company. We wanted something that could adjust to fit the way we worked - we even called it Auroragility.

You don't want to work for the methodology, you want the methodology to work for you, which is why we made those changes. This included building dashboards, describing the current situation at each point, clarifying the purposes of each meeting, trying to make processes less complicated, and expanding them where needed. For each idea, we tried to understand from the beginning what and how it would benefit the business and the organization. The biggest experiment was the PI (Program Incremental) planning where the whole company met face to face in order to plan the work for the upcoming sprints. This was a great chance to plan ahead while getting to know one another

This helped the whole team to understand the commitment they were making and the tasks they'd be working on to help achieve the agreed-on goals. We were able to talk about the purposes, the risks, and to handle conflicts and blockers across teams. It proved its success that's why we continued with it. There were also benefits from a cultural perspective, as Aurora Labs works in a hybrid remote model with employees spread across the entire country.

Creating structure

Creating the Auroragility methodology also meant implementing more structure to support employees. This included adding templates on the retrospectives for the team as well as updating templates for sprint reviews and offering training on how to demonstrate the results of something a team had been working on. We also started to do more issue tracking in JIRA to make it more collaborative for all teams. This gives more visibility into what everyone is working on.

Making these changes was a case of understanding where the company was and then looking at the big picture to see what improvements could be made to productivity while giving everyone more visibility into how things were progressing.

What's next?

Despite being new to the role, I feel I've already made an impact on the way the team works at Aurora Labs and I'm eager to continue this forward motion. We want to continue developing the visibility so both managers and developers can see what the situation is. I also want to work on the commitment to each task, so everything that we say will be completed actually is. Then, if there's the risk of something not being done, we can raise a flag as soon as possible to help us replan. This way we can be more visible with our clients, too.

With a friendly culture that understands the importance of the agile way of working, I'm also keen to keep learning from others in the company. We have a daily where different people update each day. For example, the CEO will share his updates then we might hear from the marketing team, VP R&D, or the HR team — this helps us all stay in touch, even when working remotely.

This means everyone in the business understands what others are working on but it also helps to make the team feel more connected to one another. Everyone has their responsibilities and is able to own them, which is something I really like. There's a great culture and it feels like we're all truly supported by one another as well as the management team.

Key takeaways

In my time as scrum master, I've been able to implement some important changes at Aurora Labs while learning a lot about the industry and what it's like to work with a startup.

  • Learn the company culture - This helps you better understand where you fit in and where you can make an impact.
  • Get to know your colleagues - This encourages better collaboration and makes it easier when you need to turn to someone for advice - especially important when working from home!
  • Meet the CEO regularly - Understanding the CEO's goals both for the business and for the organization - this enables better planning, but also reminds you what you're working toward.
  • Get external help - Lean on the skills of an external consultant or coach for expert feedback on the processes you are implementing.
  • Make the method work for you - Be agile yourself! Agile methodology should work for you and the business, not the other way around.

I look forward to continuing this work and looking for additional ways to make Aurora Labs more agile, all while stepping up to the challenge provided by a fast-paced startup.

AI insights increase software quality in the automotive industry

While many people associate artificial intelligence (AI) in the automotive industry with autonomous vehicles, it's actually a powerful tool that's driving software development too. I recently joined James Carter and David Fidalgo on the Byte Off Podcast to talk about the impact AI is having across the industry.

AI has the potential to improve the outcomes for quality and support engineers during the automotive software development process. Here at Aurora Labs, we're using AI to recognize patterns in the behavior of the software, as how it behaves indicates how it runs. By identifying these patterns, you can begin to predict when and how a piece of software might fail before it actually does.

Using the right AI tools (such as Vehicle Software Intelligence) we're able to help car manufacturers find problems in their vehicles before they cause failures. This allows them to focus on improving quality instead of running around trying to fix problems.

The challenge in using AI tools across these areas is correctly identifying when it's appropriate. A lot of people see this technology as a silver bullet that will fix all sorts of problems, but it's actually most powerful in areas where the inputs are unknown or the variables are great.

This is why it's so often associated with autonomous driving because the technology has to be smart enough to understand that every road, every car, and every tree looks different and still be able to identify them as such. The technology needs to be able to recognize these patterns and learn from the information it is fed.

The shift left

What we're seeing now is a shift left, which means we're starting to use AI much earlier in the development process. The idea is to catch problems earlier as this makes them easier to fix, keeps costs down, and saves valuable time. It's similar to the process of building a house. If you find a problem in the construction of the walls and identify this early on, it's much cheaper to fix the issue than if the issue had been discovered after the house was complete.

The shift left in the automotive development world is similar. It's about moving your quality tools and insights earlier in the process so you're not leaving everything until the end. Fixing issues early on is much less expensive than patching them with over-the-air updates or worse, recalling your vehicles.

There are other trends influencing this shift. Both the move to CI/CD and agile software development methodology play a role. This means a car that might have been designed over six years, for example, can now be designed in a much shorter period. With these shorter development cycles, it's vital manufacturers are testing their software early enough in the process so as not to cause delays further down the line.

Another trend is the move toward the software-defined vehicle. With the software disconnected from the hardware platform and any specific model year, there needs to be more focus on the quality of that technology as it's driving so much within a vehicle - even across different models and generations, in some cases. With this, CI/CD, and agile workflows, there's an openness to try new AI tools to improve quality and give actionable insights early on at a much lower cost than you might have with more traditional development methods.

Testing the modern vehicle

Because of the complexity of a modern vehicle and now, the option to add features via a subscription, existing testing methods become far more difficult. If you're trying to write test scenarios for every permutation of variation and in every configuration, you can very quickly get to a point where an engineer physically can't write all these tests - and you certainly don't have enough time to run them, even with automation tools.

AI algorithms, however, can monitor the behavior of the software as it's being run and pick up on deviations automatically. Without any manually defined thresholds, the AI is able to detect changes in behavior. This allows engineers to focus their attention on what is changing and what could potentially affect the vehicle quality and performance.

This benefits both end-users and OEMs. The customer gets their update or subscription feature immediately and can trust that the new software isn't going to affect something else in the vehicle. Manufacturers, on the other hand, are able to improve quality quickly and more affordably while keeping customer satisfaction high.

Artificial intelligence is a powerful tool and something the industry is becoming increasingly open to. If you'd like to find out more about automotive software quality assurance take a look here.

Why is NAND Endurance a Problem for Car Manufacturers?

All the code running advanced driver assist systems (ADAS), infotainment, performance features, and other core functionalities of a vehicle is stored on the flash memory of NAND chips. You'll no doubt have seen how the current chip shortage is affecting the automotive market, but there's another challenge surrounding these chips: memory endurance.

A flash device has a finite lifespan based on how many times you can erase and write to its sectors. When a vehicle receives an update, this is exactly what’s happening; the old code is erased and replaced with the updated code.

This means these chips are beginning to degrade and will, eventually, fail. This causes a knock-on effect throughout a vehicle with glitchy systems, non-functioning features, and potential safety issues, too. So, it's in the best interests of car manufacturers to look for ways to either improve endurance or minimize the number of times this data is erased and written.

Why is NAND endurance a problem?

Worn-out NAND chips can cause a whole host of problems, something Tesla came up against in 2019. Failing chips caused rearview camera problems, as well as the absence of turn-signal chimes and other audio alerts. The problem affected 159,000 vehicles and the recall report showed failure rates of NAND chips of over 30% in "certain build months" with failure trends accelerating after three to four years in service.

Tesla has been at the forefront of this technology for a long time so these problems begin to give us some insight into what other manufacturers could experience in the future. With so many vehicle features relying on these chips, there's the potential for multiple car systems to be affected -- from audio signals, as with Tesla, through to a vehicle's ADAS features.

Where safety-critical features are involved, manufacturers don't have the luxury of waiting around to see how long a component will last. A costly recall may be required as soon as the flash memory starts to show signs of failure. At the very least, a worn-out chip could prevent new erase-write cycles, which would make it difficult to deliver critical firmware updates.

Regular updates can cause problems

All modern vehicles require updating. For some, these updates focus on infotainment systems but, for more technologically sophisticated vehicles, the updates cover firmware and safety-critical features too. Whether delivered over cable in the garage or remotely over the air, current software update technologies erase the existing code before writing the updated code over the top.

With flash memory only able to withstand a limited amount of these erase-write cycles, the NAND chip begins to wear out. This leads to problems with the car's features, as Tesla saw in 2019.

Changing the way vehicles update could be the solution

In order to prolong the lifespan of the flash memory, manufacturers must minimize the number of write-and-erase cycles each chip goes through. Aurora Labs has created an update method that, instead of reprograming the entire flash, writes an update to the next free space on the memory. This can be done up to 20 times before the data needs to be erased, prolonging the life of the chip.

On top of this, it minimizes downtime of the vehicle as the software can still run on the old code that's contained on the chip while the update is happening. This improves the update process but could also increase the lifespan of a single NAND chip by 40 times when considering the potential for version rollbacks.

If you consider the recall report for Tesla that showed some chips failing within three to four years, this means Aurora Labs' Auto Update could increase the lifespan of a single chip from three years up to 120.

As vehicles become more technologically sophisticated, automakers need to consider the lifespan of these small parts. Using software to solve hardware problems isn't new but it's something that needs to be addressed sooner rather than later in order to avoid recalls and safety issues further down the line.

If you'd like to find out more about Aurora Labs' Auto Update feature, please get in touch.

The Difficulty of Safely Certifying Software-Defined Vehicles

Certifying a vehicle was once a one-time process. A manufacturer would build the car, get it certified, and then it wouldn't need certifying again until the next generation or refresh when significant changes were made. In the wake of the software-defined vehicle era, things get a little more complex as it's easier than it's ever been for manufacturers to update the core functionality of a vehicle via an over-the-air (OTA) update.

Take Tesla, for example, any Tesla owner can choose to add self-driving capabilities to their car. All they need to do is buy the update and the new functionality will be delivered wirelessly. This fundamentally changes the functionality of the vehicle.

The challenge with software-defined vehicles

Currently, ISO and other functional-safety certifications look at the software in a vehicle, taking into account cybersecurity and safety concerns; however, the existing process doesn't match up to the complexities of modern cars. To provide new functionality and features, these vehicles go through potentially dozens of software updates during a single-vehicle generation. This has the potential to change how the vehicle works, as well as how different systems interact with one another.

While high-end electric brands such as Tesla lead the way with OTA updates, other manufacturers are catching up. These software releases are going to happen more and more frequently, especially as we move toward large-scale EV adoption. There's a challenge in safely certifying vehicles that are updated so regularly. While many of the changes may be small, there's no knowing how those small updates could affect other systems in the vehicle.

For example, a small change to the braking system doesn't just affect how the car stops during normal driving, it'll affect the ABS, the emergency braking, and even the adaptive cruise control; all these systems are intertwined. For example, Tesla rolled out an update for its Autopilot system, improving how it used regenerative braking — this had knock-on effects throughout the vehicle.

The importance of continual certification

While manufacturers are often able to show what elements of the code have changed when recertifying a vehicle, they lack deeper visibility into the impact the code has on different systems. So while an update to the braking system might need to be recertified, there needs to be a way to also look at how that affects advanced driver-assist features and other systems — especially those that could impact safety.

This is why, in this era of software-defined vehicles with dozens of systems that work together, we need to move to a process of continual certification. Running impact analysis on a change allows us to see much more deeply into what has been affected by an update. With this kind of insight, the items that need to be recertified can be, but the rest of the vehicle won't need to go through that same lengthy process.

Continually recertifying vehicles within a model year, potentially after every core functionality update, will also increase safety. There's even the potential to use technology to constantly run impact analyses and send these directly to the regulatory body. However, the current process of certifying a vehicle is lengthy and wouldn't scale to support this kind of ongoing certification.

The challenges

Currently, different regulatory bodies have different certification processes but, typically, these involve manufacturers submitting documentation detailing all the changes. There's no standard to these documents so every single OEM may submit something different. Following this, there are multiple rounds of comments, meetings, and questions to determine how significant the changes are to the vehicle and whether or not it needs to be retested. This works, to a point, but when considering continual certification for vehicles that are regularly updated, this method won't work at scale.

AI can provide a solution

There is perhaps a solution to be found in artificial intelligence, specifically Vehicle Software Intelligence. Aurora Labs has developed a tool that runs continual impact analysis on a vehicle to understand the changes that have been made. This not only shows the affected interdependencies from an update but can also flag other issues that could impact recertification -- such as superfluous code.

Aurora Labs' Auto Validate allows both manufacturers and regulatory bodies to gain a deeper understanding of any functionality changes and how they affect other systems. This not only gives developers more insight but could drastically speed up the certification process.

The future

Innovating in this way and moving to a continuous certification process isn't currently a priority for manufacturers but, as vehicles become yet more advanced, this is something that will need to be addressed. Vehicle Software Intelligence presents an opportunity to bridge the gap between OEMs and regulatory bodies while improving safety and compliance.

While the certification process is unlikely to go through any radical changes any time soon, there's an opportunity here for technology to simplify a lengthy process. The regulatory process was designed for vehicles that only change once every few years -- but things have changed. Vehicles are now complex computer systems on wheels that can be updated at the touch of a button with zero downtime for the driver. The fast-paced nature of this technology requires a new way of working that will only scale when supported by AI tools such as Vehicle Software Intelligence.

Part 2 – Challenges of Using Software as a Revenue Generation Tool

While both new car manufacturers and traditional OEMs are embracing software updates as revenue generation tool, there could still be some bumps in the road ahead. There's no doubt that adding additional features and upgrades to a car after the initial sale can drive new revenue streams (see part one of this blog series) for carmakers, but all great opportunities come with their share of challenges.

Currently, the cost and complexity involved in overhauling legacy plated processes mean some OEMs have forms been slow to adopt OTA updates as a method of delivering feature and firmware upgrades. However, legislation, new regulations, and maintaining the user experience could present an equally sizeable barrier.

Cost

As vehicles become more sophisticated, the cost to keep them up-to-date with the latest features could easily spiral out of control. Some methods of updating a vehicle require huge amounts of data to be stored and transmitted for every update. Both full-image and binary updates could see costs run into the millions for cloud storage alone. To provide the type of updates demanded by consumers, OEMs need to look for ways to reduce these costs.

One method is through Line-of-Code Intelligence, which doesn't fully overwrite the flash storage in a vehicle. Instead, it just updates what is necessary and writes to the next available space on the chip. This can help to reduce costs (as there's less data to store and transmit), as well as improve the experience for the end-user.

User experience

As a driver, there's nothing more frustrating than jumping in your car only to find out you have to wait for the vehicle to update before you can use its core functionalities. If manufacturers are to deliver new features to a vehicle in order to increase revenue, the experience needs to be seamless.

Because, full-image and binary updates erase the previous code, the driver would need to wait while the car is being updated. In most cases, this shouldn't take too long, but it's far from convenient. Line-of-code updates are a little different, however, and allow the driver to continue on as normal with no break in how they use their vehicle. This is because the previous code isn't erased, so the old version of the software can continue to run while the update is being delivered.

Safety concerns

While many manufacturers are currently able to make updates as needed to their vehicles, some experts have safety concerns, arguing that novelty and performance features could cause problems. Even where OTA updates are delivered to improve safety, the argument is that these may not have been adequately tested in the same way they would be at the point of manufacture.

Legislation is coming into place that references OTA updates, how they're tested, and the impact of new safety features. This is on top of insurance validity concerns around changing the functionalities of a vehicle, especially when manufacturers offer free trials of different services or those on a subscription.

The UN has already established a set of rules around cybersecurity and software updates. The WP.29 rules (R155 and R156) will come into force in the EU in July 2022 and will be mandatory for all vehicles by 2024. While many of these rules surround cybersecurity, they're also focused on "providing safe and secure software updates and ensuring vehicle safety is not compromised."

As well as safety and security around software updates, WP.29 will also cover these areas:

  • Managing vehicle cyber risks
  • Securing vehicles by design to mitigate risks along the value chain
  • Detecting and responding to security incidents across a vehicle fleet

Manufacturers will need to comply with these regulations for all features delivered with the vehicle, as well as those delivered via an update. While it will take time for these regulations to come into force fully, it's important that manufacturers take steps to ensure they are fully and satisfactorily adopted.

Insurance increases

Many insurers consider new features delivered via an OTA update to be a modification to the vehicle. This could lead to an increase in insurance prices or, at worst, render the cover invalid. We all know to report modifications to our insurer, but the rules around new features delivered over the air aren't quite so clear.

Recently, UK insurer LV did a U-turn on its policies after charging Tesla owners a premium following routine software updates. It told the consumer association Which?: "We now recognize that it isn't fair to expect customers to contact us for every update, so as a result of this valid challenge, we are changing our approach."

With no existing set of rules for insurers, each will decide its own approach to these updates. This could make life difficult for consumers and could impact how car manufacturers deliver updates in the future.

Vehicle Software Intelligence as a solution

While there may be challenges ahead for OEMs, the opportunities for revenue generation are too good to ignore -- especially in this rapidly evolving market. One solution that could ease the pain of these safety and regulatory challenges is artificial intelligence, specifically Vehicle Software Intelligence. This makes the update process more straightforward for car manufacturers by minimizing the size of update files, reducing costs, and giving accurate visibility of a vehicle’s entire software system -- supporting auditing and compliance efforts.

While software-defined vehicle manufacturers are leading the way when it comes to delivering OTA updates, legacy OEMs are catching up. In fact, more than 20% of industry experts expect software sales to account for at least 10% of carmakers' sales by 2027. The road may not be as smooth as some may hope but it's the early adopters that will reap the rewards in the years to come.

Find out more about how Vehicle Software Intelligence could help your business here.

Part 1 – Software will create profitable new revenue streams for OEMs

For many years, software has been an enabler for hardware, but, increasingly, it's becoming a source of revenue for automotive manufacturers. In fact, according to McKinsey, data-driven services could create up to $1.5 trillion in additional revenue for OEMs.

Profits on a new car are ridiculously tight, with many manufacturers making just 13-21% gross profit margin (GPM) on a car sale. These margins are squeezed even tighter thanks to supply chain disruption; increased steel, energy, and logistics costs; and more R&D spend. Compare this to the software industry, where the average GPM is 72.31%, and it's no wonder OEMs are turning to software to boost margins. Many are already using over-the-air (OTA) updates to deliver new features to vehicles even after the initial sale.

One of the most well-known examples of this is Tesla offering acceleration upgrades to its vehicles with a simple update -- for a fee, of course. But there's potential here for other brands to follow suit, and many already are.

Giving more to customers

Adding new vehicle features via OTA updates will not only drive revenue to manufacturers but can improve customer satisfaction too. Drivers can add new functionality to their vehicles as need dictates or if they want to customize a used vehicle to their needs. But this is so much more than updating satellite navigation or adding new infotainment features, software gives OEMs the chance to update the functionality of the car itself through firmware updates.

This often includes small upgrades to improve the performance of a car, as in the case of Tesla. The Polestar 2 is another example, however, it gained 67 horsepower from an update to the powertrain ECU, with a retail price of $1,125. This is a fantastic indication of what we can expect in the future as OEMs begin to use software as a revenue generation tool.

Tesla is used to monetizing these upgrades and does so with great success. For example, for $10,000, you can buy the full self-driving package for your car. Tesla makes this easy for customers and simply delivers these new features through an OTA update. This, essentially, activates the existing hardware enabling drivers to make use of Tesla's full suite of autonomous capabilities.

BMW is another manufacturer that's offering more to its customers through remote updates. Owners can choose to add a range of digital services to their vehicles -- either for a one-off price or a monthly subscription. You could add active cruise control, adaptive suspension, or BMW Drive Recorder -- this automatically activates in the event of an incident but can be used to record beautiful surroundings and road-trip memories at the touch of a button if you choose.

New revenue streams

It's not just BMW, Tesla, and Polestar monetizing these updates. Stellantis recently announced a strategy that will build on existing vehicle capabilities to transform how customers interact with their vehicles -- the company is predicting this strategy to generate 20 billion euros in incremental revenue by 2030.

Stellantis CEO Carlos Tavares said: "Our electrification and software strategies will support the shift to become a sustainable mobility tech company to lead the pack, leveraging the associated business growth with over-the-air features and services and delivering the best experience to our customers."

Manufacturers have plenty of options when it comes to monetizing functionality upgrades via remote updates. One-off fees add permanent features to a vehicle but the subscription model allows OEMs to create recurring revenue streams. General Motors is already using this to add new functionality to older models. Owners of around 900,000 vehicles built from 2018 can add navigation to their infotainment system for just $15 a month.

The opportunities are endless here, especially as manufacturers look beyond their infotainment systems and to the firmware of a vehicle. Updating the features of a car -- such as safety systems, performance, or self-driving capabilities -- is where the real money lies.

OEMs aren't shy about their plans to make money from these additional features. Markus Schafer, head of research for Mercedes cars, told CAR magazine: "We're aiming for an additional 1bn euro by 2025 to be added from packages and services that we're selling over the air. Of course, we want to provide features and new experiences to our customers, but also ultimately to do additional business in the future after we've sold the vehicle. That's going to be more and more important."

For consumers, OTA updates mean personalization for their cars, allowing them to add all the features they require to an otherwise standard used car. Adding these functionalities enables OEMs to continue earning from older vehicles while promoting brand loyalty among used car buyers. While software-defined vehicle manufacturers are leading the way, legacy OEMs are catching up. In fact, more than 20% of industry experts expect software sales to account for at least 10% of carmakers' sales as early as 2027. The road may not be as smooth as some may hope but it's the early adopters that will reap the rewards in the years to come.

 

Read Part 2 of this series here

When was the last time you used a map book to get to where you were going?

Why is hardware a major cost driver in vehicle software updates?

In the world of software updates - why are we even talking about hardware?

For in-vehicle applications, the software that runs different functions is installed directly into the vehicle (unlike cloud-based applications that we're getting accustomed to in other industries). There are as many as 100 mini computers or electric control units (ECU) in a vehicle, each consisting of both software and the hardware components that run them. When manufacturers plan the ECU components, one of the key decisions is what size memory to allocate. Chip makers price memory storage (Flash and RAM) differently based on capacity - and the price difference between the different sizes is tremendous.

To ensure the cost of a vehicle is as effective as possible, manufacturers purchase memory chips based on the size of the program file. Since flash sizes are standard and program sizes can vary, often not all memory is utilized. For example, if one program (measured in compiled binary code) is 13MB, the closest chip size is 16MB, leaving 3MB free for software image growth over time.

To conduct software updates, a sufficient amount of available space must be available on the memory drive.

Why do OTA solutions require available memory space?

There are a few different methods in the market today to send update files to vehicles. Some send the entire program all over again, and some send the delta or only the changes between the current and the new version of software. However, to assure a failsafe update process, all the different methodologies have one thing in common: you don't erase the existing version until the new one is safely installed. Verifications of the newly updated version and the enablement of a safe rollback to the previous version is required to ensure a smooth transition.

This means that at any given time, an ECU's memory must allow for both the original program as well as the complete update file simultaneously - making the need for memory greater than the original size of the program. Using our earlier example, a 13MB application taking up 81% footprint may not leave sufficient memory space for additional update files.

The initial (now legacy) OTA update methods require a full second/redundant bank of flash memory, also known as A/B memory. Whether a full image or binary diff update, it is very costly and requires creating dual partitions in the endpoint memory and doubling the available memory. In this manner, the updated software version image is written to the second partition, and if it fails, the system can revert to the previous version that is located on the first partition.

How much can a little flash drive possibly cost?

Conservative prices for 512MB NAND flash chips that might be used in domain controller architectures cost about $8 each, but a manufacturer may produce several million vehicles annually with multiple (3-4) domain controllers, TCU, Head-Units and gateways. Even in architecture with smaller discrete micro-controller ECUs that use 2, 4, 8 or 16 MB flash memory chips, they typically still cost $1 or more each, and a vehicle may have 100 ECUs that are required to be updateable. If a manufacturer builds 10 million vehicles a year that require double banks of flash memory, the additional cost could easily surpass $1.5 billion.

To read more details about potential costs of OTA updates, visit the Guidehouse Insights' Cost Consideration Guide.

Aurora Labs' additive update files - the cost-effective alternative

Aurora Labs' Line-of-Code Intelligence technology has revolutionized the way that updates files are created. Unlike existing binary diff update methods, Aurora Labs' Auto Update integrates into the development toolchain and automatically analyzes and identifies changes in the lines of code. With this intimate understanding of the software, we are able to generate the world's smallest update files (6x smaller than other binary diff methods and a fraction of the size of a full software image). Our small update files are additive update files that are written to the next free space on the existing flash memory. As many as 20 update files can be written to the existing free space avoiding the need for additional memory drives or even external memory 'boosters'.

To learn more about our OTA technology, visit the Auto Update product page.

The silver lining of complex vehicle software QA

Can you imagine manually QA'ing a system that looks like this?

We certainly cannot. And based on the recent Automotive Software Survey conducted by Strategy Analytics, neither can most industry developers:

 

The advent of technological advancements into vehicles represented by complex, interoperable and interconnected in-vehicle software systems present not only development challenges, but drastically increase the complexity of quality control.

As car makers and Tier-1 suppliers gear up to solve and support new quality control measures, we'd like to shine the light into some opportunities that rise from the growing complexities.

Opportunity: Implement continuous QA for vehicles that are on the road

As the automotive software industry follows software development trends, the pace is escalating using rapid development methodologies. The shift from the longer and phase-oriented development process to continued development is necessary to keep pace with the exponential growth of in-vehicle software to support electronic and software-led vehicles. The maturing software development methodologies from 'v development to CI/CD create opportunities for ongoing and continuous quality control that will in turn increase customer satisfaction and prolong the value of the vehicle. Survey respondents also reaffirm this notion looking at upcoming new model development.

Opportunity: Conduct proactive QA to alleviate costly recalls

As car makers implement continuous monitoring and tracking of 'on the road' vehicles, they gain the power to be proactive about threats, software friction or new bugs. This can present a huge opportunity for a major paradigm shift in the automotive industry: vehicles can detect problems before they cause system failures and remotely fix them OTA in-lieu of lengthy and costly recalls and garage base visits.

The importance of proactive actions is strongly resonating within our survey respondents as well with 93% agree with this notion:

Opportunity: Accelerate the implementation of AI-powered Vehicle Software Intelligence

As outlined in the example above, ECUs and system capabilities within a vehicle are interrelated and have direct effects on each other - necessitating a deep and current understanding of their relationships and dependencies. Utilitrends in the Automotive Software industryzing AI and dynamic monitoring, Vehicle Software Intelligence technologies will be a must-have for developing, monitoring and testing connected vehicles. These technologies will not replace existing development and QA processes and tools, but rather be an additional intelligent layer to bring static data to life and give insights at the line-of-code resolution. At Aurora Labs, we strongly believe that Vehicle Software Intelligence is the key to solving development challenges and the early onset of these technologies will help the OEMs innovate and iterate faster.

To see additional , read the full Automotive Software Survey report.

Why car manufacturers are designing the software and not just the car

It used to be the mechanical details of a vehicle that made it stand out. Buyers wanted to know who had the best engine, which four-wheel-drive system was superior, or simply which was going to be the most reliable in bad weather. While these things still matter, times change, and OEMs are looking for new ways to differentiate themselves from the competition.

In the last decade, there's been a clear shift in the automotive landscape. We're seeing new propulsion types, the rise in autonomous abilities, and a level of connectivity that feels like it's straight out of science fiction. Consumers want to know if a car will park itself, whether an over-the-air (OTA) update will make it go faster, and which new advanced driver-assistance systems will keep them safe behind the wheel.

All these innovations rely on one common factor; software -- and for new energy vehicle (NEV) startups, in-house development has been crucial from day one. Legacy manufacturers are racing to catch up.

Software as a competitive differentiator

With the average vehicle containing around 150 million lines of code, the software makes up a large part of a car's value -- dictating new features such as gesture control, self-driving abilities, and voice interaction. With the likes of Tesla and NIO leading the way with software, many other automotive OEMs are looking for ways to bring their development in-house. This would not only improve time to market but also offer clear differentiation from competitors.

These changes won't happen overnight, though, as digital transformation of this scale takes time. Our recent Automotive Software Survey showed the majority of respondents predict that 10-25% of vehicle software will be produced in-house by mass-market manufacturers in 2025.

Consumers expect OTA updates

Most buyers think of Tesla when it comes to OTA updates for a good reason. The electric-only manufacturer has been building these capabilities into their cars since the launch of the Model S in 2012. Other automakers have struggled to keep up, though most now offer some form of basic OTA updates.

What is still setting NEV companies, such as Tesla and NIO, apart is the type of updates they offer. Most manufacturers can update the software on the infotainment systems but those leading the charge can also administer OTA updates to the safety-critical systems. This means being able to make adjustments and upgrades to more complex systems such as braking, steering and ADAS. Legacy manufacturers will struggle to do more than update the navigation and infotainment systems with their current development processes and OTA update solutions.

Data from Statista shows the value of the worldwide OTA update market could be as much as $7.5 billion by 2025, meaning it's not an area automakers can ignore. To keep up with consumer demand and not be left standing by NEV powerhouses, OEMs are looking for ways to quickly increase the capabilities of their OTA updates and launch new features. Bringing everything in-house is the clear solution but this will take time, meaning OEMs will continue to work with tier-one suppliers.

Managing the transformation

Using these suppliers is still necessary for most manufacturers but the benefits of in-house software development can't be ignored. It can help keep costs down, fast-track delivery, and protect against cyber vulnerabilities but there's a solution for OEMs who still need to outsource some elements of their software development: Vehicle Software Intelligence.

The key to working with suppliers is visibility. It's important to understand the bigger picture of inter-dependence and operability between elements developed by different vendors. Aurora Labs' Vehicle Software Intelligence is an AI layer that is used by OEMs in their software development efforts and the way they manage suppliers.

One challenge, for example, is software dependencies. When OEMs rely on third parties for their development needs, it's easy to lose sight of how different systems hang together. With a Line-of-Code Intelligence solution, manufacturers to get a better view of the system as a whole. This allows developers to keep an eye on the thousands of inter-related functions and capabilities to better understand the potential effects of new features and conduct OTA updates with confidence.

With 77% of respondents to our 2021 Automotive Software Survey stating that the trend towards in-house software will increase, it's clear that automakers have some work to do. The development of software needs to be treated as a strategic move by OEMs that want to stand out as the demand for software-defined vehicles continues to grow.

Want to learn how to apply Vehicle Software Intelligence to your software? Contact us.