How MarineTraffic is turning a sea of information into actionable intelligence
It’s hard to comprehend how much data flows through the MarineTraffic system each day. A staggering half a billion positions are received daily from stations around the world, up to one position per ship, per minute, which is filtered down to 30 million positions. In 2016 alone, that amounted to 700 gigabytes of data, which is held in the company’s vast databases.
This is just the tip of the iceberg, says Alex Charvalias, whose responsibility it is to turn MarineTraffic immense wealth of raw data into meaningful information useful to each customer’s needs. Alex joined MarineTraffic two years ago as a Product Manager for data intelligence products and now also manages the entire data team in his role as Head of Data.
An indication of how important data is to the future of MarineTraffic comes from the fact that while the company has doubled in size since Alex started work, the data team has grown at double that rate, to become four times larger over the same period.
As the company continues to move forward, Alex will be in charge of employing cutting-edge technologies such as artificial intelligence and machine learning to analyse and extract the greatest possible value from the company’s huge data resources.
“Our work with AIS data goes back a long way,” Alex explains. “AIS is a system that produces a lot of noise from all the position reports, which are often duplicated by multiple stations, for example. The most difficult and complex aspect of utilising AIS data to monitor the shipping industry is understanding what you see: acknowledging the limitations of the system and cleaning up the data to make sense of all this information. Over the last decade, our core focus has been to develop our understanding of this data and refine our algorithms so that we can extract information of value to the six million people who use the platform every month.”
Establishing a reliable picture of each vessel’s position is just the start of the puzzle for Alex and his team. They use their increasingly sophisticated algorithms to map the AIS position data against their extensive knowledge of the shipping industry, in order to identify operational events happening around the world.
For example, AIS data only tells them when a ship stops moving. But by referencing its position against the geographical geometries they have defined, they can understand whether a ship is laid up, at an offshore terminal or even work out which commodity it is loading or unloading, by establishing which berth it is in at port.
Alex and his team’s ability to cut through the noise and mine the data as required, already provides with the actionable information they are looking for; whether that’s giving ship owners more accurate information on supply and demand; helping suppliers to understand traffic in their areas to target new potential clients; or giving financial analysts data to model global commodity flows.
MarineTraffic expertise in AIS data analysis uniquely positions it to respond to the evolving needs of the market and individual consumers, to provide them with increasingly valuable information. To stay ahead of the curve, the team are focussed on developing increasingly sophisticated analytical techniques to extract the maximum possible value from the data they’re processing each day. Embracing new technologies for big data analysis is key to meeting these challenges.
The team are already using such techniques to help them move beyond understanding just what happens in each port to capture every single event that happens anywhere in the sea, calculate expected vessel arrival times and identify specific patterns of movement, such as search and rescue operations or ship-to-ship transfers.
“For centuries, we have relied on rumour, hearsay and other incomplete information to try to understand how small corners of the industry work, say transporting crude oil or cruise ship maintenance, for example,” Alex explains.
“But developing a system that captures the phenomenal complexity and scale of the entire international shipping industry and global commodity flows, from shipper to consumer, has always been out of reach – until now.”
The pioneering big data analysis technologies that Alex’s team are exploring offer huge potential to help MarineTraffic create an increasingly vivid picture of global shipping. Developing the desired top-to-bottom view of the entire industry will take years of considerable effort from the whole team. But Alex is excited about these future challenges.
“We are investing a lot in creating the skill set and finding new solutions to attach a meaning to anything visible through AIS,” he explains. “We have unrivaled experience in making sense of this data, but we don’t sit still. We keep trying to find new ways to increase our understanding in order to ultimately help the maritime ecosystem become more efficient and transparent – which has been MarineTraffic vision from the very beginning.”