Data Transfer Project – An Open Source Initiative For Data Portability

Data Transfer Project (DTP) works to improve data portability by allowing users to transfer data and files directly from one online service provider to another. Introduced by Google, Facebook, Microsoft, and Twitter, it is an open source initiative that encourages participation of as many service providers as possible. Once fully implemented, DTP will reduce the infrastructure burden of both users and service providers and increase the number of service providers offering portability.

How does Data Transfer Project work?

Data Transfer Project is made up of three components:

1. Data Models:

These are the canonical format that has been put in place to bring a common understanding of how the data transfer should be done. It provides methods of converting data from different service providers to forms that are usable by the system. Data should be in the right model when being transferred between different companies. Each service provider should adopt interoperable APIs to allow data to flow smoothly between service providers.

Data models are categorized into industry groupings, which form a vertical hierarchy. The groupings are based on the type of data to be transferred. This data can be photos, contacts, emails or music. Each grouping has its own Data model that helps to enable seamless data transfer of the relevant file.

Image Source: Data Transfer Project White Paper

2. Company-specific adapters:

Company-specific adapters are classified into two; Authentication Adapters and Data Adapters. Company-specific adapters are implemented outside service providers’ core infrastructure but can be developed by the provider himself or third party that works to enable data transfer from one provider to another.

Data adapters translate service provider’s API into Data Models used by DTP. They include exporter that translate provider’s API into the Data Model and importer that translate data model to provider’s API.

Authentication Adapters are the piece of code designed to enable the consumer to transfer data from one provider to another.

3. Task Management:

The Task Management Libraries are designed to handle background tasks such as securing data storage, calls between adapters, retry logic, pagination management, rate limiting, individual notification and failure handling. There are a number of task management libraries that help to transfer data between different providers.

Importance of Data Transfer Project

There are numerous reasons why users want to transfer data from one service provider to another. DTP is designed to make it easier for customers to interact with their data as much as they want. Direct-data transfer is managed by the receiving service provider without the need for the customer to see the DTP branding or infrastructure at all. Here are some of the examples of how service-to-service portability is important:

1. Users what to be able to initiate data transfer directly from a social media platform to their phone book services.
2. Users, who may not agree with the privacy policy of a particular music service provider, would want to stop it immediately but may not want to lose their playlist. All they want is to transfer to another music service provider.
3. Customers may want to export data from legacy providers who are going out of business to large and stable companies without having to worry about data format issues.
4. Users working on architectural projects may want to transfer all files to shared storage systems where clients can access them. DTP is designed to allow such users to transfer hundreds of large project files without straining their bandwidth.

Why do we need Data Transfer Project

Many companies are adding functionalities of downloading data into their system. Such functionalities are great steps toward empowering users to be able to control their data fully. However, implementation of data download functionalities has its own shortcomings that include the following:

Low bandwidth: downloading and uploading data can be time-consuming if you have low bandwidth. This sometimes makes DTP too expensive and often impractical to implement.

Incompatible Formats: various companies have come up with unique data structures, which are in machine-readable form. These data formats are not compatible with one another making it hard for users to re-import back data that had been imported.

Discoverability: sometimes it is very hard to discover the data service providers that allow data transfers.

Data Transfer Project work to solve these problems and make it easier for users to transfer data directed from one service provider to another. They are working to ensure that the burden of bandwidth and compatibility is solved by the service providers and not users. There is also the alternative approach of DTP that proposes that each service provider build direct transfer capabilities.