How the Mega Players conquered Discovery

I have enjoyed reading the report from the first NGI Workshop. Overall it seems to support the vision and the working areas targetted by the current consultation. I am concerned, however, that it did not mention the term "Discovery" although "Discovery and identification tools" is listed as a specific technology area in the consultation.

Interestingly the executive summary starts by explaining the rationale behind the overwhelming success of the internet due to openness and flexibility of the data transport layers. I think that nobody today will disagree that the standardization of smart network protocols and the DNS system for "Discovery", was a decisive factor in creating this scalable success.

In contrast, the subsequent section of the summary describes how the commercial power of the internet has shifted towards large ISPs and cloud service providers. It mentions how people are becoming frustrated because of the rise of powerful Internet mega-players. This is elaborated further under Issue#2, suggesting a number of possible advanced technology options to remedy this problem "and return to a certain equilibrium between the big Providers (at the Internet core) and the vast activity of local networks of users, business networks, markets and communities"

This power concentration, however, should not at all come as a surprise. While the scalability and vendor independance of the transport layers was a result of thoughtful engineering, the problems now mentioned are directly related to the fact that similar basic "Discovery" considerations for the application layer were totally ignored when the internet was populated by ordinary persons and companies around 1990. Without focus on "Discovery" of companies and people - the single most important task performed by the mentioned "mega players" - any amount of technology research will only lead to secondary changes to the status quo.

The Importance of Native Discoverability

Digital identity management is all about how the real identities of people, companies, and things can be reliably associated with data describing their qualities, services, and needs/preferences. Now, 25 years after the commercialization of the internet, we still do not have a standardized model for discovery of digital identities and their services. A major reason for this is that we still think in terms of the industrial age economy, rather than on what is needed to enable a globally scalable information age economy.

An example of this missing foresight is the new eIDAS regulation which focuses on assurance related to login and digital signature between parties already introduced to each other. This may be appropriate for the local village economy (where everybody knows one another) and the industrial economy (with news papers or search engines as intermediaries). But to enable a scalable information age economy supporting the EU vision of a "Single digital Market" it is of paramount importance to enable persons - and in particular - companies to make themselves as well as their services and qualifications natively discoverable on the internet.

Up until now the lack of native discoverability has been compensated by platform companies such as Google, Amazon, Ebay, and Uber stepping in to provide discoverability of various types of service providers. Unfortunately this model requires that both sellers and buyers have to be known by and typically registered with the same infomediary. This has now become detrimental to both cost, choice, privacy, and convenience.

100 Bn Euro per year just to be discovered?

Analysis of revenues for various platforms and other online services show that the cost of basic discovery is typically in the range of 10%-30% of the traded value, while seller/buyer assurance/reputation/KYC is in the order of a few percent and basic transaction costs are down to tiny fractions of a percent. On an EU basis this corresponds to added discovery costs in excess of 100 billion EUR pr. year merely considering e-commerce related activities.

Search engines that are charging businesses to be discovered by consumers worked really well in the early internet days, when only a few dozens of vendors were online in many product and service categories. But in a global market with maybe 100.000 relevant vendors for a specific type of product or service, there is a significant discrepancy between the respective interests of the search engine and of the consumer. Whereas the search engine looks for the highest fee from a vendor, the consumer may prioritize fast delivery, low price, and adherence to the consumers personal preferences and privacy expectations.

Targeted Advertising necessitates much richer profile information about you!

Additionally, the high discovery costs associated with search engines makes it vital for vendors to target their advertising efforts very precisely to the intended customer segments. This can only be accomplished by gathering much richer profile information about potential customers than otherwise needed, which may again cause conflicts with the upcoming privacy regulation.

In market areas, where there are multiple platform vendors, such as in the taxi business, consumers as well as drivers are increasingly inconvenienced by the need to administer at least a handful of apps to reach a proper selection of their potential counterparts. With native business service discovery, any customer app can discover any cab - and vice versa - just as we are used to reach other people and companies even if they are using other banks and telcos than ourselves.

It is promising to see that the need for alternative business models focusing on the consumer is now being articulated by persons and organizations covering a wide spectrum of society: From Silicon Valley business gurus as John Hagel talking about the "Trusted Customer Advisor" to the EU Data Protection Supervisor sharing his opinions on "Personal Information Management Systems"

Implementation of native business service discovery will support the competitiveness of such new business models by providing easy and free access to product and service information. Bringing both the data and the analytics to "the consumer side", further provides the possibility to combine "Big Data", "Privacy by Design" and eventually "AI" for consumer decision making. Vendors success will then be based on actual product qualities rather than on how much data they have managed to gather about specific customers.

How might the NGI initiative bring back balance?

Even more generally it could be discussed whether we are ready to manage the Internet of Things, as long as we have not been able to handle Discovery and Identity appropriately for the two classes of things that own and ultimately should govern all other classes of things: Companies and persons. Hence I see the task of NGI as a matter of designing appropriate scalable architectures supporting the commercial use of the internet for the single digital market - rather than launching a range of more or less coordinated projects with an aim to "return to a certain equilibrium between the big Providers (at the Internet core) and the vast activity of local networks of users, business networks, markets and communities"

Fortunately, due to its already advanced business registers and coordination between these, the EU is in a favourable position to lead the implementation of such NGI initiatives. I have written more about possible benefits and scenarios for a distributed business service discovery solution at bedreid.dk. Establishing native discovery for individual persons should also be considered an important task for NGI, but is a much more complex challenge due to privacy considerations as well as the individuals usually much more dynamic touchpoints with the internet. There are, however, several projects, e.g. MIT's Solid project, that may serve as inspiration for upcoming NGI efforts.