Navigating Complexity: The Challenge of Building Reliable Automotive Software

With vehicles featuring more advanced driver assist systems than ever -- as well as autonomous capabilities -- it's no wonder software has become the most important element in a modern vehicle. As a result, automotive software is becoming more complex than ever, with more lines of code, modules, and components. This means the challenge of maintaining quality and reliability becomes even more pronounced.

Traditional software development practices are struggling to keep pace with the growing needs of today's vehicles. Many developers continue to use outdated methods, such as repetitive and time-consuming testing strategies, in an attempt to improve quality and reliability. The new era of software-defined vehicles demands an innovative approach to meet the need for comprehensive testing and rapid iteration.

Artificial intelligence as a quality assurance tool

Artificial intelligence (AI) tools are emerging as key allies in automotive software development -- especially when it comes to quality assurance. At the design level, AI tools can provide tested and proven blueprints, reducing the need to build every project from scratch. This significantly simplifies the development process and saves crucial time and resources. Furthermore, AI can provide a framework that maintains consistency and speeds up the development process.

When it comes to testing, AI's potential is equally transformative. These tools can identify specific tests to focus on based on the updated software functions and their interdependencies, which helps optimize test selection. The end result is a more efficient testing process that helps support the creation of reliable software and updates.

Aurora Labs' Auto Detect adds a layer of AI to the testing process to give actionable insights into which tests have the highest probability of failure. This means that when a new version is committed to the software repository, Auto Detect will optimize the order of test runs by selecting the tests to run first, ensuring a faster time to failure. This speeds up the software testing process while still ensuring all essential functions are tested.

The same tool can also improve quality by ensuring 100% test coverage. With hundreds of millions of lines of code and complex interdependencies throughout a single vehicle, manually trying to achieve 100% test coverage is a monumental task. Auto Detect uses Line-of-Code Intelligence to deliver detailed insights into the coverage of your tests, allowing developers to see what's covered by the testing scenarios -- and, crucially, what's not.

Culture change

In an industry as established as automotive, change can be difficult to implement, especially when it comes to ingrained development processes. However, the need for process innovation is as pressing as the need for software innovation. Traditional development models, such as the sequential production line model, are often inadequate for managing complex software projects.

An agile approach embraces small iterative steps and anticipates potential issues. Tools such as those developed by Aurora Labs are already proving to be game-changers in this area. For example, the automotive industry is familiar with the V-model development methodology, where customer requirements and corresponding tests form the two arms of the "V" with development at the base.

Traditionally, aligning these two aspects has been a manual task. However, AI tools can now read and understand documents, interpret the context, and map out the connection between the requirements, the software, and the corresponding tests. In this way, AI is not just working side by side with software development but is becoming an integral part of the process by identifying areas for improvement and ensuring compliance with regulations.

The road to reliable automotive software development lies in striking a balance between software innovation and process refinement. AI tools can undoubtedly catalyze this journey, but their success hinges on the ability of developers to weave these tools into an efficient, scalable, and innovative development process -- one that is prepared for the rapid changes defining the future of the automotive industry.

If you'd like to find out more about how Aurora Labs' AI tools can support your automotive software development, download the whitepaper here: Five software development challenges in automotive and how AI is addressing them.

Automotive Saftey (r)Evolution

On September 30, 1955, police, ambulance, and fire crews arrived at the scene of a horrific two-car collision at a desert intersection in Cholame, California. Medics found that the driver of the Porsche, rising star James Dean, had been thrown from the car and killed instantly.

Following an investigation into the collision, it was determined that Dean hadn't been wearing a seatbelt. If he had, he most likely would've survived the crash. This high-profile death raised public awareness of the importance of seatbelts and ushered in the age of automotive safety.

Not much changed in the coming years in terms of new safety features in vehicles but more people did start to wear seatbelts. It was only when electronic control units (ECUs) started to become commonplace that manufacturers began to introduce safety features such as ABS, ESP, airbags, and more recently, lane-keeping assist, Forward Collision Warning (FCW) and other advanced driver assist systems (ADAS).

History of the ECU

General Motors introduced the first electronics system into a vehicle in 1978. By 1981 all GM vehicles contained an engine control unit that helped manage fuel use and power within the vehicle. In the early 1980s, hybrid digital systems became popular with other manufacturers, too, these were able to measure and process inputs from the engine to yield preset output values. This ROM system, as it was known, was one of the first tunable systems.

By 1991 almost all US and Japanese manufacturers has abandoned carburettors in favour of fuel-injection systems controlled by microprocessors.

Now, all ECUs use a microprocessor that processes engine inputs in real-time. They're much more robust than other systems, especially as an engine begins to wear. As well as lending themselves well to tuning, these systems also enable more sophisticated safety features.

ECU-based safety technologies

Many vehicles now have a dedicated ADAS ECU that draws information from the vehicle's cameras, lidar and radar systems, and inertial measurement units, as well as map data. This enables various safety features, including:

  • Lane-keeping assist
  • Blindspot warning
  • Adaptive cruise control
  • Automatic emergency braking
  • Hill descent control
  • Lane change assistance

How to safely create an ECU

In order to power these advanced safety features, an ECU needs to be safe in itself. In 2014, for example, there was an incident where a software defect caused unintended acceleration with drivers unable to apply the brakes.

In order to avoid similar issues that could impact the safety of a vehicle, it's important that software is thoroughly tested. With more than 100 million lines of code and interdependencies that span every single vehicle system, this is too much for a human developer to work on alone. The use of artificial intelligence in the testing process is vital when it comes to obtaining 100% test coverage in order to uncover potential defects.

It's also important for developers to understand how each function in the software relates to one another. For example, a small change or update to the software controlling the braking system will affect more than how the car stops during normal driving -- it'll also impact ABS, emergency braking, and even adaptive cruise control, as all these systems are intertwined.

One of the main conclusions from an investigation performed by the Barr Group that led to the creation of a safety standard was that "testing is not enough to establish safety". Today, we understand that even these standards are needed to be enhanced.

Line-of-Code Intelligence is an AI-based tool that maps complex automotive software systems in order to understand the interdependencies of functions that might otherwise seem unrelated. This helps developers create safer ECUs from which to run advanced driver assistance features.

Regulating these complex systems

In 2011, the International Standards Organisation (ISO) created ISO 26262. This framework helps to identify the potential risks of software and hardware failure in a vehicle. As part of this, there are specific Automotive Safety Integrity Levels (ASILs) that can be assigned to a safety requirement and its potential hazards. These are determined by a series of classifications based on the likelihood of a hazardous event, the severity of a potential injury, and the controllability of a driver to prevent or mitigate that injury.

Depending on those factors, the safety requirement is given an ASIL ranging from A to D. The most safety-critical systems are ASIL D, and these have the most stringent testing requirements.

The latest version of ISO 26262 was released in 2018 and extended the scope from passenger cars to all road vehicles. Automakers were quick to adopt it in order to make their driver assistance systems as safe as possible.

Automotive safety has come a long way since James Dean's tragic crash, but there's still a way to go to ensure modern systems are as safe as can be. If you'd like to explore more about how artificial intelligence can improve automotive safety, find out more about Aurors Labs' technology here.

Continuously Better — a Recipe for Winners

Making devices, processes, and even people continuously better is not a new idea. Case in point -- we were watching a documentary about Henry Ford last week and one of his main business principles was 'good isn't good enough', and he continuously made improvements to an already revolutionary factory floor. He implemented processes and technology -- mainly the conveyer belt - to make sure building the Model T was more seamless, bringing automotive parts to employees instead of employees going to find parts. Fast forward almost 100 years, and the same principle is executed by successful companies.

The browser wars were won because Google ensured Chrome was always being improved with seamless updates increasing speed, enhancing security and introducing new features. In the automotive world, where software is now a driving force, vehicle manufacturers are continuously making the consumer experience better with over-the-air software updates. The problem is they are not always seamless.

For example, a friend recently received a letter from his car manufacturer explaining that an update to fix the infotainment system was available. The options were to take the car to the dealer or to update the vehicle on his own. A link to instructions on how to do the update himself was in the text of the letter. Going to the noted website, he landed on a YouTube page with a video of how to do the update. When he went to his vehicle to follow the instructions -- they were totally incorrect, and the UI in the video didn't even match the UI in the vehicle. Not very seamless.

Tesla is always used as an example of how to best do seamless over-the-air updates offering new features and functions that consumers will look forward to and enjoy. On the other end of the spectrum, there are those that think updates are "Big Brotherish" and should not be allowed at all.

However, continuously better will always win. But, unlike mobile phones and laptops, 'continuously better' for the vehicle requires a great deal more effort on technology testing, quality assurance, third-party certification and regulation. The generation coming up in the world expects their vehicle to mimic their phone, and they want the same user experience. The generation building these solutions today is responsible to make sure 'continuously better' keeps the next generation safe while simultaneously meeting their expectations of personalized and satisfying user experiences.

Ghosts in the machine: why does software misbehave?

"There have always been ghosts in the machine. Random segments of code, that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul."

In the movie I Robot, based on the book of the same name by Isaac Asimov, Dr. Lanning coins the term "Ghosts in the machine" and goes on to explain why robots behave in an unexpected way and not as they were programmed.

These ghosts aren't limited to sci-fi. In fact, they're something we deal with every day in software development. As technology has become more complex, we're faced with deviations that mean software doesn't quite behave in the way we expect it to -- just like Dr. Lanning's robots.

We're not quite at the point where autonomous vehicles could develop their own will but it's true that computers, robots, vehicles, and other programmed machines create such ghosts of unexpected behaviour. Understanding why these glitches happen is a vital step in reducing their frequency and handling them when they do show up.

Let's explore some of the common reasons why software deviates from its intended behaviour.

Untested combinations of events

Testing a new vehicle is a very complex task. A vehicle model may have a high number of variations depending on the feature set, country of destination, and other parameters. This results in a variety of different versions of the vehicle software for a single model year, creating complex testing scenarios.

Despite a rigorous testing process, which includes specific combinations of events, caveats still exist. This is because it's difficult to predict all of the different scenarios a vehicle will face as there are so many variables that could affect the software at any one point.

The use of artificial intelligence in the testing process is a potential solution to this as it can help come up with combinations of events a human might not think of. The AI can create new scenarios based on the expected interactions of the code. This helps developers find new edge cases they may not have come across during a manual testing process.

 

Testing doesn't always reflect real life

As well as an untested combination of events, there's also the chance that a test suite might not cover a real-life situation. Tests might cover a very specific and narrow band of scenarios that can be predicted and controlled in laboratory testing, but life out on the road is often unpredictable -- especially when you take into account weather conditions, road type, number of passengers, etc.

After production, continuous monitoring of the software behavior, while the vehicle is on the road, can give information about real-life situations. This gives the tester a more accurate view and insight into the running code instead of a birds-eye view. AI technology can support this as the car is in use.

Glitches are often incorrectly written software

What the world refers to as a computer glitch or a software bug is often an incorrectly written piece of software. This can cause unforeseen behaviour that could lead to a software update or recall.

The reason for the incorrect software may vary. It's not because of bad developers, but because writing software for a vehicle is incredibly complex and can incorporate a lot of factors that can collide with each other. Perhaps a developer or architect neglects to take into account a specific input or measurement that may affect the behaviour of the specific function or module.

As vehicles transition to a set of computers (ECUs) on wheels, it's clearer than ever how real life can throw up surprises that just can't be replicated manually in the lab. Mistakes happen, making it vital that automakers adopt more testing and debugging capabilities. Data also plays a role here and it's crucial to be able to receive as much information as one can throughout the software lifecycle, all the way from development to maintenance.

Incorrect data

With incorrect data from the network, a car's software might behave strangely. This is because it's expecting specific types of data so it can react accordingly. If the data is incorrect, the car will be unable to respond in the manner it was originally designed. The reason for the incorrect data could be faulty flash memory, a sensor malfunction, or a bit error in a message on the network.

Some mitigation mechanisms are in place to overcome the problem of misinformation due to bit error -- such as code correction algorithms, CRC data validation algorithms, ECC flash integrity checks, and more. More types of data range limitations and checks are suited to verify the correctness.

Although these types of mechanisms are in place, even with the "correct" data, an error may arise from an unfamiliar combination of information. This could be something that seems so illogical to the human eye that it would have never been considered during the testing process. So even though the code is written correctly and was tested successfully, an error can still occur.

Data reception issues

A problem with data reception can cause lags in a system, which consequently causes operational delays. This can also be fatal if a safety system doesn't engage in the time it should. Such a scenario may cause brakes not to work when they should or airbags that don't deploy in time to ensure occupant safety.

Latency and lag in the propagation of data over the network are issues that are hard to overcome and may pose a grave danger to the vehicle, passengers, and bystanders.

Million Lines of Code -- Information is Beautiful

Modern vehicles are software on wheels, and they're getting more complex all the time. The average vehicle currently has 100 million lines of code (LOC), but this is forecasted to grow to 600 million LOC for a Level-5 autonomous vehicle. With more LOC, the chance of encountering ghosts in the machine becomes even greater. Understanding how these glitches occur and what vehicles are commonly recalled for can help us solve these problems before they get out of hand.

Leaning on artificial intelligence technology to support the development process is key when it comes to automotive software. Presenting new ways to debug and get a clearer view of what's happening in real-time is becoming imperative in order not just to develop a better, safer vehicle but also to shorten the time from when a software bug occurs to the point of finding a solution and deploying it to the cars on the road, incorporating it with the ability for fast deployment is even better.

Glitches don't always present a safety concern, but they should be treated seriously. Instead, they manifest as small issues that can cause frustration for drivers. Reducing the effect of these glitches on the end-user is vital for manufacturers. Even in cases where a glitch does appear, an over-the-air update is often enough to fix the problem. It's also important that automakers are able to quickly roll back the software to a previous version should a glitch appear following an update.

If you'd like to learn about how Aurora Labs can help you identify potential issues and improve the testing process during development, book a demo here.

 

Speed up Software Testing with Artificial Intelligence

On a complex piece of automotive software, testing newly deployed code is vital. With so many interdependencies between each function, those tests ensure any changes don't negatively affect other areas of the system. However, with the amount of code and its complexity increasing with every build, running every single test for every version of the software can take hours.

While this is a necessity for the system software before it is released for production or before OTA updates are to deployed, running all the tests for daily builds while still in development is hugely inefficient -- especially if only a specific function has been updated.

Identify all affected systems

Even when a new version of the software seems to only update one specific function, it's important to identify which other systems could be affected. The Line-of-Code IntelligenceTM technology developed by Aurora Labs maps all interdependencies to better understand the effect of any changes on other functions within the vehicle.

 

Eliminating test execution redundancy

To identify these dependencies, Aurora Labs' Auto Detect adds a layer of artificial intelligence (AI) to the software testing process. This helps to detect exactly which tests are needed for each software function, enabling the tester to focus on and only run the required tests, eliminating the need to re-run all tests every time a new version is committed to the software repository.

For example, if a system has 100 tests, Auto Detect will be able to select the right ones based on exactly what's been changed by an update -- including which other functions might be affected by the new code. This means that of those 100 tests, only 20 might need to run. This saves valuable time by ensuring only the necessary tests are performed.

By adding this AI layer to your development process, it's possible to decrease the time it takes to run tests. On top of the increased efficiency, you improve quality by ensuring dependent functions are also tested alongside those that have been directly updated -- without the need to run redundant tests.

If you'd like to find out more about how Auto Detect can improve the automotive software testing process, find out more here.

Success comes from innovation

With technology and automation developing at an extraordinary rate, accelerated even further by the effects of the pandemic, traditional automotive manufacturers are working hard to bring innovation into everything they do. The most prevalent trend in 2022 is autonomous vehicles (AV), closely followed by connectivity and electrification. Perhaps unsurprisingly, nine out of the top 10 automotive innovation trends are technological.

Automotive OEMs (original equipment manufacturers) are in a prime position to instill a culture of innovation more deeply within their businesses. It's something that's so pervasive within the industry that they can and should be taking inspiration from that atmosphere, adopting new technologies and ways of working, and bringing in new people in order to keep things fresh.

Innovation in automotive right now

According to Forbes, the automotive world is set to completely transform in the next two decades, much as it did in the late 19th century. At this time, the world's major cities were awash with manure as the use of horse-drawn vehicles reached its peak, but by 1912 Henry Ford had resolved to solve this issue with the motor car.

Another dramatic shift of this type seems feasible when you consider the speed and breadth of innovation in this industry. Electric vehicles (EVs) are now commonplace, and Deloitte's global EV forecast shows a compound annual growth rate of 29% over the next decade. Additionally, EV sales growth is expected to expand from 2.5 million in 2020 to 11.2 million in 2025, and 31.1 million by 2030.

Gartner has reported that the automotive electronics sector will experience the biggest semiconductor compound growth rate up to 2024, at 9.3%. Modern vehicles have around 8,000 semiconductor chips and over 100 electronic control units; these currently carry over 35% of the total vehicle cost, but that is expected to rise to 50% by 2030.

Software-defined vehicles

Then there are the vehicle manufacturers that have had technology and innovation at their core from the beginning. Software-defined vehicle companies like Tesla are taking a different approach and are constantly thinking outside the box. Its EVs are some of the fastest in the world, it's working on transformative energy projects to help limit fossil fuel demand, and it even considers elements like how professional drivers will be affected by autonomous self-driving. That in itself is innovative.

While more traditional OEMs are embracing a similar shift, their approach is fundamentally different. The automotive industry has been at the forefront of technological innovation since it started but with the rise in software-defined vehicles, there's a need for more innovative out-of-the-box thinking than ever. The key is to build a culture of innovation within the a company, whether it's been around for 100 years or just 10. This means embracing new technology, leaning on artificial intelligence tools, and looking for innovative ways to stand out in a market that's more competitive than ever.

Challenges

According to Forbes, some of the newer 21st-century vehicle manufacturers are challenging -- or even overtaking -- the major established players. Ford ($53bn), BMW ($62bn), and GM ($82bn) have Tesla ($64bn), Nio ($70bn), and BYD ($65bn) nipping at their heels, meaning traditional OEMs have to move quickly as the automotive industry continues to evolve.

The biggest challenge lies in having to become, essentially, a software company. Many OEMs have traditional mechanical production company DNA, and turning that into something much more software-focused is a difficult thing to pivot to. Despite this, companies such as BMW, Porsche, Hyundai, and many others have been able to embrace a culture that doesn't just look at ways to innovate when it comes to their products but also in their manufacturing methods, over-the-air-updates, their sales processes and more.

Investment in AI tools -- such as Vehicle Software Intelligence -- infrastructure, and employee training will all be key for traditional manufacturers when it comes to building a culture of innovation.

How to build a culture of innovation

We've established that the main difference between traditional OEMs and the more modern ones is that the latter started as software companies, and the former is taking steps to pivot their approach. Overcoming this challenge starts with getting the right people on your team. This is what creates the new kind of thinking that's required to keep up with modern automotive demands. Other ways to build a culture of innovation include:

  • Employee training and workshops that focus on new ideas
  • The adoption of cutting-edge tools that improve processes
  • Embrace AI and other technologies that can increase automation and free up employees for tasks that need a human touch
  • Adopt agile processes to speed up the delivery of software and updates

It's also important to know when to innovate. Consumers don't necessarily want a vehicle that looks and drives like it's straight from science fiction, but perhaps technology can improve the driving experience in more subtle ways -- such as in advanced safety features or vehicle upgrades delivered over the air.

Additionally, if AV is the biggest trend in automotive right now, then AI is an ideal area to focus on. In fact, AI and machine learning are important across the entire value chain -- not just when it comes to driverless vehicles. These technologies can improve time to market, development processes, and quality control. Software-led innovation is an opportunity that allows OEMs to maintain competitive advantage and growth. To learn more about Vehicle Software Intelligence and how it can solve your own challenges, contact Aurora Labs today.

Improve Quality with 100% Test Coverage

Quality control is a vital part of vehicle software and it's even more vital when you consider the different sources that may be contributing to the code base. Code can come from your internal development team as well as through open-source software and third-party developers. While much of the code might seem stand-alone, this isn't the case and every new line could have a significant impact across the system.

These interdependencies present a challenge for quality assurance (QA) teams. Even when additional code might seem straightforward, there could be multiple dependencies that need to be tested as part of the QA process. Line-of-Code IntelligenceTM supports both developers and quality teams by delivering detailed insights into the coverage of your tests.

The road to 100% test coverage

In something as complex and as safety-critical as vehicle software, it's vital you have 100% test coverage. With so many interdependencies, this is often hard for a human to do alone. Auto Detect uses Line-of-Code Intelligence, which as the AI sets run, will be able to map all the software functionality paths and see which are being tested. With this visibility into your code base, you can begin to see what's covered by tests and, crucially, what's not.

In building this picture of the code base as a whole, the AI layer will give you greater visibility into test coverage as well as the behavior and performance of the software as the tests run. It's from this information that it's clear to see which functions have been activated with the current test suite as well as those that haven't.

Seeing the functions that haven't been activated allows your development team to see where additional tests might be required - or, potentially, see instances of superfluous code. This insight into a rapidly changing code base can help you achieve and maintain 100% test coverage reducing the risk of releasing untested software to production.

 

Improving software quality with AI

With an efficient path to 100% test coverage, you can improve the quality of your software. With all dependencies covered across numerous functions, you'll increase software development and functional safety compliance. This removes hurdles to productivity and can greatly improve time-to-market for new features.

Proper testing of software is vital in any industry but as vehicles continue to increase in complexity in a heavily regulated sector, 100% test coverage is crucial. To find out more about Line-of-Code Intelligence and how it might help your business, read more and book a demo of Auto Detect here.

Will open standards transform the automotive industry?

The automotive industry is at a pivotal point in its evolution. With software-defined vehicles transforming the way cars are built, we're likely to see innovation in this area increase over the coming years. This will affect the way we buy cars as well as how new features and updates are delivered.

OEMs and suppliers are starting to use open-source software in the development of a vehicle, and its use will continue to grow, but open standards and APIs are going to disrupt the industry in new ways. This is something I've seen happen in finance, which has led to a rise of new businesses entering the market, shaking up a mature industry. The same could happen in automotive.

The building blocks of automotive software

Going forward, manufacturers will likely lean on open standards to build their own platforms -- much like using Lego bricks one by one to create something custom. This means they'll be able to take a component from an existing piece of software and connect it to other software from tier one suppliers. From this, they can build their own architecture while improving interoperability and enabling new avenues for innovation.

The use of open standards has numerous benefits to the automotive industry as a whole. These include:

  • Fewer limitations: With open standards, manufacturers won't be locked into a single technology, giving them a wider choice of suppliers to work with.
  • Improved interoperability: A set of standards makes applications more functional and interoperable, streamlining product development.
  • Less chance of becoming obsolete: File types in proprietary applications may become obsolete, and are often hard to convert. Open standards protect against this.
  • More chances to innovate: With more suppliers working to the same standards, this gives manufacturers the chance to be more innovative in the way they build their software.
  • Competition can flourish: With fewer barriers to entry new suppliers and, potentially, manufacturers, can enter the market.

There are already standards and initiatives in automotive that support interoperability, such as AUTOSAR, ECLIPSE SDV, and SOAFEE, as well as those that govern adjacent technologies (such as the Open Charge Point Protocol for the smart charging of electric vehicles). However, there's still a lot of potential for the automotive industry to adopt more of these standards.

Taking inspiration from FinTech

I come from a FinTech payment background, an area that is flourishing thanks to open standards. Over the last seven years, finance has gone through a lot of changes with young tech companies disrupting a mature industry. These new businesses have been able to succeed thanks to open standards and open APIs.

For example, open banking standards have allowed consumers to securely connect their bank accounts with third-party providers. This allowed a raft of businesses to enter the space offering tools and products that can do anything from analyzing your finances to facilitating payments between bank accounts across different countries and currencies.

Open standards work in established markets, which is why they've transformed the finance space. The automotive industry is likely to be next as it's mature enough to withstand the disruption while, arguably, being ready for a fresh shake-up.

If the automotive industry can take inspiration from what's happening in the finance space or, at least, do something similar, we could see some big changes in the coming years. Already, open standards and open-source software are making an impact on how OEMs develop their software, but with more buy-in and as open standards evolve, there's the opportunity for increased innovation, more choice, and ultimately, better vehicle software.

Making an Impact as a Scrum Master

"You don't want to work for the methodology, you want the methodology to work for you."

I recently joined Aurora Labs as scrum master and wanted to reflect on my first months at the company. My job involves managing the agile workflow, hosting daily meetings, coaching team members, and removing roadblocks. It's rewarding work; ensuring the smooth running of feature releases, bug fixes, updates, and more across the Aurora Labs platform.

Before joining the company, I worked as a front-end developer and technical scrum master, splitting my time between the two roles. After two years of this, I wanted to transition to a 100% scrum master role and step outside my comfort zone.

I looked for a growing startup that could benefit from my agile skills but would provide the kind of challenge I was looking for. When you work for a big company, everything is clear, and everything is defined, but I wanted the chance to bring more of myself and my ideas into a business that would give me the freedom to push myself and drive things forward.

Aurora Labs seemed like the perfect fit and I was eager to get started but I knew there would be a challenge in getting up to speed with the automotive industry and a new way of working. The first task was to get to know the business, its processes, and all the technologies it uses. A priority for me was to meet the people, too, as it's important to me to understand the culture of a business and where I fit in.

 

The differences

Working in a large company has challenges and is a very different environment than working in a startup with a close-knit team. This gave me the kind of freedom I needed to make a real impact in my work.

At Aurora Labs, you have the opportunity to pinpoint an idea or change, followed by a proof of concept. From there you bring it to the relevant people to have a discussion and explain what needs to be done. This means that changes can be made quickly and you can really see how your ideas improve and support the process. This is exactly the kind of environment I wanted to be in.

Another highlight was the buy-in from the Aurora Labs team. The CEO gave me a daily time slot in his calendar to talk about the things I needed in order to succeed. It was clear that it's really important for him and for the company to be more agile.

Additional support

Around the time I started, Aurora Labs also began work with external agile coach Boaz Fine to support me and others in the business. This improved how quickly I and fellow scrum master Gil Baruch were able to get up to speed. We worked alongside Fine to brainstorm the next steps, then discuss the right way to ensure smooth changes. This was the agile team that drove the impact on the organization forward.

 

Making an impact

When I joined Aurora Labs, the company was using the Spotify agile model. It had been working well with a clear focus on scaling agile processes while emphasizing the importance of culture and network. This is just one example of structuring teams and sprints in order to get work done in an efficient way.

It soon became apparent, however, that this wasn't quite the right model for the company. We wanted something that could adjust to fit the way we worked - we even called it Auroragility.

You don't want to work for the methodology, you want the methodology to work for you, which is why we made those changes. This included building dashboards, describing the current situation at each point, clarifying the purposes of each meeting, trying to make processes less complicated, and expanding them where needed. For each idea, we tried to understand from the beginning what and how it would benefit the business and the organization. The biggest experiment was the PI (Program Incremental) planning where the whole company met face to face in order to plan the work for the upcoming sprints. This was a great chance to plan ahead while getting to know one another

This helped the whole team to understand the commitment they were making and the tasks they'd be working on to help achieve the agreed-on goals. We were able to talk about the purposes, the risks, and to handle conflicts and blockers across teams. It proved its success that's why we continued with it. There were also benefits from a cultural perspective, as Aurora Labs works in a hybrid remote model with employees spread across the entire country.

Creating structure

Creating the Auroragility methodology also meant implementing more structure to support employees. This included adding templates on the retrospectives for the team as well as updating templates for sprint reviews and offering training on how to demonstrate the results of something a team had been working on. We also started to do more issue tracking in JIRA to make it more collaborative for all teams. This gives more visibility into what everyone is working on.

Making these changes was a case of understanding where the company was and then looking at the big picture to see what improvements could be made to productivity while giving everyone more visibility into how things were progressing.

What's next?

Despite being new to the role, I feel I've already made an impact on the way the team works at Aurora Labs and I'm eager to continue this forward motion. We want to continue developing the visibility so both managers and developers can see what the situation is. I also want to work on the commitment to each task, so everything that we say will be completed actually is. Then, if there's the risk of something not being done, we can raise a flag as soon as possible to help us replan. This way we can be more visible with our clients, too.

With a friendly culture that understands the importance of the agile way of working, I'm also keen to keep learning from others in the company. We have a daily where different people update each day. For example, the CEO will share his updates then we might hear from the marketing team, VP R&D, or the HR team — this helps us all stay in touch, even when working remotely.

This means everyone in the business understands what others are working on but it also helps to make the team feel more connected to one another. Everyone has their responsibilities and is able to own them, which is something I really like. There's a great culture and it feels like we're all truly supported by one another as well as the management team.

Key takeaways

In my time as scrum master, I've been able to implement some important changes at Aurora Labs while learning a lot about the industry and what it's like to work with a startup.

  • Learn the company culture - This helps you better understand where you fit in and where you can make an impact.
  • Get to know your colleagues - This encourages better collaboration and makes it easier when you need to turn to someone for advice - especially important when working from home!
  • Meet the CEO regularly - Understanding the CEO's goals both for the business and for the organization - this enables better planning, but also reminds you what you're working toward.
  • Get external help - Lean on the skills of an external consultant or coach for expert feedback on the processes you are implementing.
  • Make the method work for you - Be agile yourself! Agile methodology should work for you and the business, not the other way around.

I look forward to continuing this work and looking for additional ways to make Aurora Labs more agile, all while stepping up to the challenge provided by a fast-paced startup.

Why is NAND Endurance a Problem for Car Manufacturers?

All the code running advanced driver assist systems (ADAS), infotainment, performance features, and other core functionalities of a vehicle is stored on the flash memory of NAND chips. You'll no doubt have seen how the current chip shortage is affecting the automotive market, but there's another challenge surrounding these chips: memory endurance.

A flash device has a finite lifespan based on how many times you can erase and write to its sectors. When a vehicle receives an update, this is exactly what’s happening; the old code is erased and replaced with the updated code.

This means these chips are beginning to degrade and will, eventually, fail. This causes a knock-on effect throughout a vehicle with glitchy systems, non-functioning features, and potential safety issues, too. So, it's in the best interests of car manufacturers to look for ways to either improve endurance or minimize the number of times this data is erased and written.

Why is NAND endurance a problem?

Worn-out NAND chips can cause a whole host of problems, something Tesla came up against in 2019. Failing chips caused rearview camera problems, as well as the absence of turn-signal chimes and other audio alerts. The problem affected 159,000 vehicles and the recall report showed failure rates of NAND chips of over 30% in "certain build months" with failure trends accelerating after three to four years in service.

Tesla has been at the forefront of this technology for a long time so these problems begin to give us some insight into what other manufacturers could experience in the future. With so many vehicle features relying on these chips, there's the potential for multiple car systems to be affected -- from audio signals, as with Tesla, through to a vehicle's ADAS features.

Where safety-critical features are involved, manufacturers don't have the luxury of waiting around to see how long a component will last. A costly recall may be required as soon as the flash memory starts to show signs of failure. At the very least, a worn-out chip could prevent new erase-write cycles, which would make it difficult to deliver critical firmware updates.

Regular updates can cause problems

All modern vehicles require updating. For some, these updates focus on infotainment systems but, for more technologically sophisticated vehicles, the updates cover firmware and safety-critical features too. Whether delivered over cable in the garage or remotely over the air, current software update technologies erase the existing code before writing the updated code over the top.

With flash memory only able to withstand a limited amount of these erase-write cycles, the NAND chip begins to wear out. This leads to problems with the car's features, as Tesla saw in 2019.

Changing the way vehicles update could be the solution

In order to prolong the lifespan of the flash memory, manufacturers must minimize the number of write-and-erase cycles each chip goes through. Aurora Labs has created an update method that, instead of reprograming the entire flash, writes an update to the next free space on the memory. This can be done up to 20 times before the data needs to be erased, prolonging the life of the chip.

On top of this, it minimizes downtime of the vehicle as the software can still run on the old code that's contained on the chip while the update is happening. This improves the update process but could also increase the lifespan of a single NAND chip by 40 times when considering the potential for version rollbacks.

If you consider the recall report for Tesla that showed some chips failing within three to four years, this means Aurora Labs' Auto Update could increase the lifespan of a single chip from three years up to 120.

As vehicles become more technologically sophisticated, automakers need to consider the lifespan of these small parts. Using software to solve hardware problems isn't new but it's something that needs to be addressed sooner rather than later in order to avoid recalls and safety issues further down the line.

If you'd like to find out more about Aurora Labs' Auto Update feature, please get in touch.