Migrate from Entity Read Files

Stay organized with collections Save and categorize content based on your preferences.

Entity Read Files (ERFs) are JSON representations of a partner's campaign objects made available through Google Cloud Storage.

ERFs were deprecated in June 2021 and sunset on October 31, 2024. ERFs are no longer generated. Use the Display & Video 360 API to retrieve Display & Video 360 resources.

This guide discusses how to migrate from Entity Read Files to the Display & Video 360 API by:

• Giving an overview of the differences between the two interfaces
• Comparing ERF tables to API services
• Providing guidance on entity retrieval through the API
• Acknowledging existing data gaps
• Presenting a mapping of all ERF fields to comparable API resource fields

Overview

When migrating from ERFs to the Display & Video 360 API, there are a number of key differences to consider, including:

• Data freshness. ERFs are generated daily and in bulk while the API retrieves the most up-to-date version of a resource.
• Resource structure. The API uses different JSON structures than ERF to represent the same resource types. Some resources, such as public targeting settings, may use a different ID space.
• Retrieval method. The Display & Video 360 API only enables the retrieval of resources individually, in paginated lists, or through BigQuery data transfers, in contrast to the raw JSON files provided by ERF.
• Scope. As opposed to ERFs, which are scoped by partner ID, most API resources are scoped by advertiser ID. Resources included in responses are limited to resources within that scope.

ERF data representation in the API

Entity Read Files are separated into "Public" and "Private" tables. Public tables provide information that is available and applicable to all users, such as targeting values. Private tables provide data that is specific to a partner, such as creative or line item resources.

The Display & Video 360 API does not use this dichotomy, instead making all of this information retrievable through various services and using different JSON structures. This section compares the information provided through public and private ERF tables to that made available through Display & Video 360 API resources and services.

Public information

ERF public tables provide reference materials for users to use when interpreting the targeting settings of their retrieved private resources and assigning targeting through a subset of Structured Data Files (SDFs) versions uploaded through the UI. These reference materials are the same for all users and consist of a numeric ID, used for mapping, and more descriptive details, such as a display name.

When using the Display & Video 360 API, targeting reference information can be retrieved through the targetingTypes.targetingOptions service. Similar to the public tables, this service provides the IDs and details of targeting options for a specific targeting type. Refer to our existing Set Targeting page for a code example demonstrating targeting option ID retrieval.

Public tables and SDFs

Before SDF v7, Entity Read Files and Structured Data Files use the same ID space for targeting settings. If you are an SDF user using ERF public tables for interpreting or assigning targeting settings using SDF, you can instead download this reference material in CSV format through the Display & Video 360 UI.

Starting with v7, the ID spaces used by a subset of Structured Data Files columns have been updated to decouple SDF from the ERFs and further align with Display & Video 360 API. See the v7 migration guide and reference documentation for more information.

Private resources

ERF private tables provide a daily snapshot of the current settings of private resources owned by a partner. Due to the sheer volume of resources that can be created under a single partner, these files can grow to be very large and difficult to download and process.

In the API, each private table has a corresponding service that provides endpoints for retrieval and management of that resource type. Resources can be retrieved in bulk using each service's respective list method. The JSON structure for each resource is different in the API compared to ERF, utilizing different field names and shared resources.

Certain information available in the ERF representation of a resource, such as a resource's assigned targeting settings or a channel's sites, are represented in the API as children of the original resource and must be retrieved through additional API requests.

Entity retrieval in the API

Display & Video 360 resources can either be retrieved through direct API requests or automatic imports into BigQuery.

Direct API requests

Each resource type is retrievable through a different API service. Resources can be retrieved individually or in bulk using the appropriate service's get or list method, respectively. Important properties of Display & Video 360 API list methods include:

• Required scope. Unlike ERFs, which are scoped by partner, most resources in the API are scoped by advertiser. Retrieving all of a resource type, such as line items, under a partner may require an individual list request for each child advertiser of that partner. Exceptions include direct children of a partner, such as advertisers and partner-owned channels.
• Pagination. API list methods employ pagination to ensure that responses are within a reasonable size, limiting most individual request responses, or pages, to 100 resources. If the number of relevant resources is larger than the page size, consecutive list calls are required to retrieve subsequent pages of the full list response. A code example paging a list response is provided in a section of our Targeting guide page regarding retrieving available targeting options .
• Additional requests required for targeting retrieval. A resource's targeting settings are not included in its API JSON object, but are instead child resources known as assigned targeting options. These child resources must be retrieved through a separate request. For example, for each line item retrieved through an advertisers.lineItems.list request, a separate advertisers.lineItems.bulkListAssignedTargetingOptions request must be made to retrieve all targeting information.

Optimize resource retrieval

The Display & Video 360 API may require multiple requests to retrieve the same amount of information that is available in a single Entity Read File. Optimizing how you retrieve resources can help retrieve the data you need more efficiently:

• Make concurrent requests to the API. The Display & Video 360 API protects infrastructure using requests per project and requests per advertiser per project rate limits. This quota structure lets you implement a multithreaded solution across multiple advertisers that will reduce the total time it takes to retrieve all necessary resources. Although pagination requires that all resources of a type within a certain scope are retrieved through consecutive calls, retrieval of resources within another scope or of another type can be done concurrently.
• Utilize filters and order by parameters in your list calls to retrieve only relevant resources. For example, if you are only interested in line items that have been updated in the last day, you can use the filter parameter of the advertisers.lineItems.list method to only return line items with an updateTime greater than a given timestamp. This can significantly reduce the number of requests that need to be made.
• Cache regularly used IDs to avoid unnecessary API requests. Certain reference information, such as targeting options IDs and Google Audience IDs, are relatively stable and can be safely stored to avoid the need for retrieval upon every use. However, cached values should be checked on a weekly basis to account for infrequent changes or deprecations.

See our Quota Optimization guide for more information on how to access the Display & Video 360 API efficiently.

Import into BigQuery

The Display & Video 360 API BigQuery Connector lets you automatically import Display & Video 360 resource configurations directly into BigQuery on a daily basis. The configurations are stored in BigQuery using Display & Video 360 API resource design. A subset of API resources are supported.

See the following cloud documentation for more information about using the Display & Video 360 API BigQuery Connector:

• What is a BigQuery Data Transfer Service?
• Schedule a Display & Video 360 transfer
• Display & Video 360 data transformation

Known API data gaps

There are notable data gaps you might encounter when migrating from ERF to Display & Video 360 API, such as:

• Story insertion orders. Story insertion orders are not retrievable through the API and must be retrieved through the Display & Video 360 UI.
• A subset of resource fields. A small number of resource fields present in ERF objects are not available in the corresponding resources retrieved through the Display & Video 360 API.

Appendix: Mapping ERF Fields to API

Public table mapping

The tables below map the fields of ERF public tables to existing targeting types and targeting option fields in the Display & Video 360 API. Although the value of one field might map to another, that does not guarantee that they utilize the same data type, enum values, or ID space.

App Collection

Retrievable under targeting type TARGETING_TYPE_APP_CATEGORY.

Browser

Retrievable under targeting type TARGETING_TYPE_BROWSER.

DataPartner

There is no equivalent resource or fields available in Display & Video 360 API.

DeviceCriteria

Retrievable under targeting types TARGETING_TYPE_OPERATING_SYSTEM, TARGETING_TYPE_DEVICE_MAKE_MODEL, and TARGETING_TYPE_DEVICE_TYPE.

GeoLocation

Retrievable under targeting type TARGETING_TYPE_GEO_REGION.

Isp

Retrievable under targeting type TARGETING_TYPE_CARRIER_AND_ISP.

Language

Retrievable under targeting type TARGETING_TYPE_LANGUAGE.

SiteToPlacementId

There is no equivalent resource or fields available in Display & Video 360 API.

SupportedExchange

Retrievable under targeting type TARGETING_TYPE_EXCHANGE.

UniversalSite

There is no equivalent resource or fields available in Display & Video 360 API. Individual sites and apps can be targeted directly under targeting types TARGETING_TYPE_URL and TARGETING_TYPE_APP, respectively. In Display & Video 360, any app or URL can be targeted, but not every app or URL can be reported on. If you would like to remove non-reportable apps and URLs from spending, follow the instructions on the DV360 Help Center.

Private table field mapping

The tables below map the fields of ERF private tables to existing fields or services in the Display & Video 360 API. Although the value of one field may map to another, that does not guarantee that they utilize the same data type, enum values or ID space.

Advertiser

Campaign

Creative

CustomAffinity

FloodlightActivity

InsertionOrder

InventorySource

LineItem

NegativeKeywordList

Partner

Pixel

There is no equivalent resource or fields available in Display & Video 360 API.

UniversalChannel

UserList