General Data Cleansing Methods
Often times raw AIS messages can contain errors, empty or default values, and points over land due to GPS issues or spoofing.
For the feed served up via API, we do our best to clean out some of these errors. Below is a list of some of the steps that we take to clean the data:
- Check values against the AIS standard.
- Remove position data with unavailable coordinates (91 / 181).
- Filter out messages with invalid MMSI numbers. (Ships should report 9-digit MMSIs. Base stations should report 7-digit MMSIs.)
It is easy to get the full feed, including errors, by simply attaching the
cleansed=false parameter within calls to the Messages API.
Below is a non-exhaustive list of items that we do not account for in data cleansing:
- Ships reporting positions over land
- Ships reporting an unintelligible name or destination field
The TCP feed is the rawest form we provide, which does not include data cleansing.