LINC Project Journal 5 - finalizing the driver less shuttle tests

This section is based on interviews with project participants Per Beakgaard (DTU Compute, Soren Jacobsen (Nobina), Hannah Villadsen (Roskilde University Center), and Jeppe Rich (DTU), who are all engaged in the operation end the evaluation of the tests with driverless shuttles at DTU Campus. 

The operations on DTU Campus in numbers: 

• 3 Vehicles
• more than 6 month long test period (April 19th, 2021 - November 5th, 2021)
• average commercial speed per day around  5 km/h, max speed 15 km/h
• more than 3.000 hours of operation.
• On average around 91% of the time in AUTO self-driving mode.
• more than 6.000 person-trips
• around 1.000 students, staff, and visitors at DTU joined a users panel to test the shuttles, and share their expectations and experiences
• In total there have been 665 LINC app downloads, whereof 234 are for Android and the rest is for iOS. 

In addition to operating the shuttles, the LINC project also released the smartphone sensing platform with a connected app for users. The platform uses GPS signals and Bluetooth beacons to sense the movements of buses and users. By doing this, the sensing platform can for example determine which users are onboard the bus and if there are users at the bus stop.

During the tests, data has been collected from the vehicles and the smartphone sensing platform. Data has also been collected in many other ways: surveys have been sent out to users, researchers have travelled with the buses to monitor behaviour, and interviews and focus groups meetings and a stated preference (SP) survey have been performed. Researchers have also performed video analysis of movements and human-robot interactions at crossings. 

When this journal is written, the project team is currently in an intense phase to analyse the data since operation ended November 5th, but some learnings can be made already now.

It is more difficult than one can expect to operate these shuttles

says Soren Jacobsen, Nobina. The project's experience is that self-driving technology is far from mature. Although the vehicle operates up to 91% of the time in autonomous mode, the last 9% of the time includes many challenging situations including misplaced cars and interactions with pedestrians and bikes.

The infrastructure and the environment around the vehicle have a large impact on the operation. Looking at DTU Campus it seems to be a good candidate for testing autonomous driving: it is a semi-public environment, with low traffic speeds and few cars. However, during the tests it turned out that the campus environment is very complex with its many bikes and pedestrians, cars and trucks temporarily parked along the roads and falling autumn leaves from the large trees that interfere with the Lidar sensors on the vehicles.

 The vehicles are programmed to follow their digital track and to exactly the same route over and over again. In comparison, human drivers are very good at handling exceptions, e.g. adopting the route if there is a hindrance. Such situations are more challenging for the AV shuttles to handle.

The vehicles are part of a large system, including the vehicle-to-infrastructure communication, the smartphone sensing platform, the test passenger app, the steward app, and the users and operators. Changes in one part of the system often require changes in other parts. For example, a change in the route impacts many other aspects.

From an organisational view the LINC project has been very complex: many different actors, coming from different backgrounds and carrying different perspectives. Through good spirit and good collaboration the project has achieved the cross-pollination that is crucial to be successful.

Operational costs in LINC have been huge, but we should remember that the technology is still immature. In the longer term, when the technology is mature and the market is ready for adoption, the operational costs are expected to go down and be lower compared to a traditional bus. The transition to a system with self-driving vehicles will likely still require large investments in both technology and in related processes. When the first investments are made, it is expected that marginal costs for adding more vehicles to the system will be relatively low.

One key target for the LINC project is to explore the use of driverless shuttles in on-demand service. Due to the delays in getting legal permission to operate the vehicles, the tests had to be limited to a frequency-based service only. Instead, the on-demand service was explored using simulations based on expected demand in Hersted Business park after the opening of the new light rail and testing a number of different service designs.

Surprisingly, the results showed that the frequency-based service outperformed the on-demand service in most cases. In particular, this is true for the case of longer distances (urban sprawl). Based on these findings, the research team draws the conclusion that on-demand services perform best in dense areas while in the relatively sparse areas of Hersted Business park it is likely that a frequency-based service would be the best choice. Further research is needed including a larger area of the future Greater Copenhagen Light Rail to finally conclude on which autonomous service design will be the most appropriate to implement in this specific context.

The number of returning users has been lower than expected. This means that the service was not adopted in the way that was expected at the beginning of the project. Instead, most users have been curious “tourists”. One important reason is that for most people it was in fact faster and more convenient to walk or bike instead of waiting for the shuttle. However, disabled persons or persons carrying a lot of loads found value in the service.

The shuttles also interact with road users, and pedestrians and bikes experienced that in some situations the shuttles’ behavior was very different from manually driven vehicles. This sometimes caused confusion and unexpected situations on the streets.

The steward onboard has been very important during the project. The stewards were supposed to be in the background, to mimic a driverless service. However, during the project, they have actively participated in improving the service offered, supported and informed the users, and sometimes tailored the service to meet the users’ needs. One learning is that the role of providing a good and tailored service is very important, and a steward could play an important role there even if the shuttles manage to drive autonomously.

The LINC journey is about to end. The project has developed and deployed an innovative mobility service using self-driving vehicles on public streets. A six months-long field test period has now ended, and the project is currently in an intensive phase of analyzing data and producing final insights. While still awaiting the detailed results from the project, two main learnings can already be made.

Operating self-driving vehicles in a service that users are willing to adopt is a challenge. With the current state of technology, the functionality is too limited to provide a meaningful service to the users if the vehicles are to operate on public streets and among other road users. Given the current state of technology, it is not obvious that self-driving shuttles can provide a meaningful service at a university campus. With the learnings from the LINC project, it would now be relevant to start from the outside in and identify the gaps and the pain points in current collective mobility, and then see how self-driving shuttles could be a part of the solution. The smartphone sensing platform could be a good tool to use.

Providing a mobility service is not only applying advanced technology. It is important to recognize that it is a complex socio-technical system that involves infrastructure & environment, social patterns & users, and technology. For example, the project members realized general weak points in the infrastructure, and they got the opportunity to explore movement patterns and mobility demand through the smartphone app. The fact that the technical test did not deliver the expected service instead leveraged learnings in other fields. One conclusion is: it is important to perform tests to learn!

The project is currently closing the activities and putting together the final results in reports and scientific articles. The project partners will also continue communication with politicians and decision-makers at regional and city levels. 
Given the relatively immature technology of the self-driving shuttles in combination with the Danish legislation, which still is very complex and restrictive for autonomous operation, it is not a feasible path to aim for regular operation of the self-driving vehicles at this point. Instead, Nobina aims at further demonstrations and pilots with the shuttles used in LINC. Such pilots would create valuable knowledge for future deployments. Another, and complementing, path forward is that the project's simulation for one light rail station is deployed to more stations if the Greater Copenhagen Light Rail’s Board of Directors finds it meaningful.

At the moment another AV road test is not on the table, as the legal framework in Denmark will be being revised in 2022. The project partners have been invited to contribute with their own experiences. Parts of the project will properly be taken to new projects (for instance Horizon Europe 2021-2027) such as the smartphone sensing platform for demand-responsive services and innovative mobility services. Besides, in the post-project period the following result indicators will be monitored:

1. Transforming urban planning by providing autonomous mobility, where the aim is to integrate new collective mobility concepts into the Area Development Plans for at least two of the urban areas along the new Greater Copenhagen Light Rail. 
2. Behavioural change and citizen engagement, where the aim is to use a participative approach to support passengers to feel confident with the collective autonomous bus service.
3. Legislative and regulatory framework impact, where the aim is to use the learnings and experience from the LINC project to reduce legislative and regulatory barriers for the operation of autonomous vehicles in Denmark. 
4. Improved measurements of mobility patterns, where the aim is to use the smartphone sensing platform developed in the project to improve and enable other mobility services. One example is enabling tickets that are valid during mode changes.