Empowering Communities with Data Technologies

December 14th, 2017

The Link between AutoMat and BigDataEurope

The BigDataEurope (BDE) project was launched in January 2015. At first sight, it has a lot in common with the AutoMat project – both projects focus on big data and the transport sector. Moreover, both projects aim to help users make better sense of big data by providing platforms, such as Automat’s Vehicle Big Data Marketplace or BDE’s Big Data Integrator Platform. Both projects are now discussing how to leverage data from other sources, such as open data portals, social networks, meteo, and police, to improve their platforms and the provided services.

The AutoMat project was invited to participate in the BDE workshop in September, which discussed how big data is transforming the transport sector. The workshop was a lively event and provided much useful feedback for both projects. Afterwards, the AutoMat project kindly contributed a blogpost to BDE’s blog, which is available here. In return, this article is aimed at informing the followers of AutoMat about BDE.

BigDataEurope – The Starting Point

The growing digitization and networking processes within our society have a large influence on all aspects of everyday life. Large amounts of data are being produced permanently and, when these are analysed and interlinked, they have the potential to create new knowledge and intelligent solutions for economy and society. Big Data can make important contributions to the technical progress in our key societal sectors and help shape business.

To this end, BDE addresses seven key societal challenges. Alongside transport, the project has dedicated groups focused on health, food and agriculture, energy, climate, social sciences, and security. These have been identified as the most important societal challenges by the European Commission in its research and innovation framework programme Horizon 2020. BDE looked at the user needs for big data in each of the seven societal challenges, hosting workshops and webinars involving participants active in the data chain throughout the project lifetime.

The Big Data Integrator Platform

The aim of BigDataEurope was to develop an adaptable, easy to deploy and use solution, which would allow interested user groups to extend their big data solutions or introduce big data technologies to their business processes. This objective was achieved through the creation of the Big Data Integrator Platform.

All European stakeholders are now able to take advantage of the Big Data Integrator Platform by using a concrete methodology for producing technically sound solutions and maximising their outreach to relevant communities.
Optimising the use of big data through the Big Data Integrator Platform is expected to bring multiple benefits to the transport sector. These include:

  • Resource-efficient transport that respects the environment
  • Safer mobility
  • Better and more informed door-to-door mobility
  • Less congestion and fewer unforeseen delays
  • Improved socio-economic and behavioural understanding
  • More secure exchange of personal information
  • Forward looking policy making

Thessaloniki Pilot Site

Thessaloniki was selected as the pilot site for the BDE transport societal challenge, given that the city’s roads are already witnessing large-scale deployment of cooperative services for both passenger and freight transport at European level. The existing services in Thessaloniki have been improved through the use of big data techniques.

The pilot demonstrated the ingestion, processing and storing of data to classify traffic conditions and to make predictions. The pilot deployment was based on the Kafka, Flink, PostGIS, and Elastic Search components of the Big Data Integrator. In the pilot, the algorithms for map matching and classification were implemented in R and accessed by the Java/Flink application. Kafka has been tested as the middleware, upon which different modules implemented in different languages are integrated.

For citizens, the achievements of BDE mean new and better services, which have been improved with the forecasting capabilities based on mobility and traffic pattern recognition. In other words, citizens are able to benefit from more precise, faster, and more detailed information via improved time estimation, traffic flow estimates, road hazard detection and traffic model simulations.

The Way Forward

The BDE project will end at the end of December 2017. However, the Big Data Integrator Platform will remain accessible to interested stakeholders on GitHub. Moreover, the BDE consortium is glad to see that there are new projects that are receiving financing from the European Commission and that will continue the fight to make sense of the immense data world out there.

For more information and to get in touch with the BDE consortium, visit https://www.big-data-europe.eu/