Design Doc: Property Cache Modifications

Property Cache Modifications

Kishore Simbili, 2012-12-15

Background

MPS uses property cache to store the stable properties (like critical images) of the page. While storing the data, we use one key per url which assumed implicitly to deal with only one type of device i.e, Desktop. With the advent of new devices of different form factors (mobile, tablet, may be TV) entering into internet space, it’s the time to break the assumption. If the publisher serves different content for different form factors for the same url, then the page has very few (maybe none) stable properties to share across form factors, while the property values themselves are stable for a particular form factor.

The properties of the page could very well change with different devices. Eg., the critical images set will be different with different resolutions of the screen.

Following is the table to summarize some data to show the number of sites that serve different HTML content for a given URL(if it is redirect, we consider it to be different content).

Total sites tested	1173
Sites with same content for all UA	849
Sites with mobile html different from desktop	282
Sites with tablet html different from desktop	164

There are good number of sites which serve different content for non-desktop. But the majority of the sites still seem to serve the same content.

Objective

Design a mechanism to store the data in property cache separately when needed, and retrieved it correctly while serving. The mechanism should have the least cache fragmentation.

Design Proposal

We define UA (User Agent) groups as the set of UAs for which same HTML content is served from the origin.

The UA groups are predefined in an enum.

Enum UAGroup {
  kDesktop,
  kMobile,
  kTablet,
  ...
}

For a given URL, we read all the property pages for all known UAGroups. To facilitate this, we add a new function to PropertyCache

typedef std::list<PropertyPage*> PropertyPageList;
// Reads PropertyValues for multiple pages, calling corresponding
// PropertyPage::Done as and when page read completes. It is essential
// that the Cohorts are established prior to calling this function.
void MultiRead(PropertyPageList* property_page_list) const;

These property pages are stored in ProxyFetchPropertyCallbackCollector::property_pages_for_ua_groups_. All these lookups happen in parallel to kDevicePropertyCache lookup. Determining the UAGroup for the current request is done after kDevicePropertyCache lookup. Based on the UAGroup determined, we choose one of the property page from ProxyFetchPropertyCallbackCollector::property_pages_for_ua_groups_ to be used for the request.

If the property page for determined UAGroup is absent then we fall back on to some safe UAGroup.

Towards the end of the request, if the property page lookup failed, then

1. If the properties computed during the life of current request differs (NOTE: The properties can be compared using some distance metric. We could weight some properties like response code higher.) from the properties of the fallback UAGroup then write the new properties under the UAGroup in property cache.

2. If the properties are similar then we continue to use the fallback group.