On November 13th, the NGI Forward project (the NGI’s initiative’s Policy Lab) held an expert workshop on the topic of search and discovery in the Atelier de Tanneurs in Brussels.
This workshop brought together over 30 invited experts from across Europe to reflect on the future of internet search, and help shape the European Commission’s funding and policy agenda in this important area.
This blog discusses some of the main take-aways of this day; a longer report, informed by all the great insights we gathered during the day, will follow soon. If you are interested in being involved in these conversations, do get in touch with the NGI Forward project or sign up to stay informed here.
Search and discovery?
The way in which we order, discover and retrieve information online is one of the, if not the, key building blocks of the internet. It is therefore no surprise that many of today’s technology debates prominently feature aspects of search and discovery: from fairness in automated decision-making and recommendation algorithms, to the sustainability of the internet; from the impact of online disinformation on our democracies to centralisation in the digital economy.
But it is not just in the present that these topics are so important. New technological developments in, for example, artificial intelligence and the IoT space as well as rising hyper-connectivity blurring the boundaries between offline and online, might dramatically change how we think about search in years ahead. This workshop is an opportunity to surface some of these emerging dynamics and opportunities.
Search and discovery is a key topic on the agenda of the Next Generation Internet, the European Commission’s ambitious flagship programme aimed at building a more inclusive, democratic and resilient future internet. The purpose of the workshop was to bring together experts working on different aspects of search, across disciplines and industries, to reflect on the current state of the field, and recommend ways in which the NGI can help strengthen existing ecosystems.
Throughout the day, we focused on answering three key questions:
- What are today’s main challenges and opportunities in the space of search: what does the current landscape look like?
- What might search and discovery look like in 5 to 10 years? How are emerging dynamics reshaping this space?
- What can we the search community do to help ? What are Mechanisms to strengthen the ecosystem
Biggest challenges and opportunities in search today
At the start of the workshop, we asked all participants to share what they thought were the biggest challenges and opportunities driving development in search and discovery. From this exercise, we’ve collected a lot of very varied and in-depth insight into the current state of the space.
As a group, we distilled the discussion into five overarching umbrella topics:
- Centralisation of power: Many elements of search are dominated by just a handful of players. How do we find the kind of business models and seize new opportunities around, for example, decentralisation that might help level the playing field in this key sector of the digital economy?
Participants emphasised we should not just look at centralisation when it comes to access to data (and respective size of user bases), but take a full-stack approach, where we look at how power can be better distributed across layers of the internet.
- Sustainability and resilience: One key concerns several participants surfaced was the environmental impact of search and data storage more broadly on the planet. The budding field of green search tries to address the high energy intensity that comes with search- from ensuring we minimise computing power required to run search queries to ensuring we limit storage and duplication of data. Developments across the search space should be studied with a sustainability lens in might: emerging opportunities in, for example, object search and IoT might help make some processes smarter and more efficient, but are also likely to add new strains on the system.
One interesting insight that emerged from our discussions on sustainability was on the need to be careful when we think about making processes more decentralised in order to reduce their energy intensity. Greener or more distributed alternatives are often good in essence, but do not always scale as well as existing systems- sometimes inadvertently increasing inefficiencies rather than reduce them. Careful cost-benefit analyses are necessary before we lock ourselves into new systems.
- Creating a user base for alternatives: Though there are many alternative solutions out there, few manage to compete with the large actors dominating this space. Part of that is a function of economics, but our participants also pointed out that smaller (open source) tools often do not do a particularly good job when it comes to user experience and usability. Addressing these challenges will require a myriad of different solutions, which will be discussed in greater depth in our final report.
What all participants agreed on, however, is that there is an important role to play for the European Commission- both in levelling the playing field through setting fair rules, but also through procuring and funding alternatives, enabling them to find a larger market and find pathways to sustainability on the funding front.
- Data quality and access: trust, bias & fairness: How do we ensure search and recommendation systems base their decisions on high-quality and representative data and do not perpetuate existing inequalities? Underpinning black-box algorithms are often hard to understand and near-impossible to challenge, which can lead to the unfair targeting of certain groups, or, probably more pertinent to the field of search, steer our behaviour in directions not of our own choosing or otherwise bias outcomes (e.g. women being shown job ads for less well-paid positions than their male counterparts).
More investment in research and tools that can help us better understand or respond to these biases and inequalities is much needed (though we must not let currently hot debates about ethics in data and automated decision-making overshadow other persistent issues in the space).
- Multilingualism: The dominance of English and other major world languages on the internet means that we lose out on a lot of richness of content (which is not taken into account in search queries, for example), and exclude large groups from benefiting fully from the digital economy. The European Union, home to an incredible linguistic diversity within its borders, can play an important frontrunner role in developing a more multilingual internet.
What might the future hold?
What might the field of search and discovery look like in five to ten years? How are emerging developments in the search space, such as new possibilities in cognitive and object search, and broader social dynamics reshaping the field?
What kind of real societal problems could these new possibilities help solve? Can we, for example, make aspects of search greener or help level the playing field in the digital economy? And what new challenges might they instead surface?
Participants agreed that if Europe wants to expand its role in the field of search, it needs to address some of the key challenges we face today, but also seize emerging opportunities and technological advances in the space. Our participants pointed out that a lot of existing dynamics will only become more entrenched as the field of search expands beyond the current confines of “the internet”. Addressing economic, social and political challenges will worsen if we don’t address them now.
There are however also many very exciting opportunities and growth fields that will likely transform the field in years and decades ahead. That’s not just topics like green search, the emerging opportunities for objects to communicate with each other, and also be “found” by one another (object search) and ways of making search more serendipitous and better able to recommend us things we might not yet know about, but fit our patterns of interest (rather than linear recommendations).
It’s also important to not treat the field of search as a vacuum: emerging dynamics in other technology field will interlink and expand the possibilities we currently have at our disposal. Think for example of the previously mentioned IoT space, but also advances in 5G, which will make continuous, real-time connectivity, discoverability and communication possible, or AI and Machine Learning.
Where do we go next?
After a hugely insightful workshop, we are not done with this work. Our upcoming report will summarise the key insights the group surfaced during the day, and will also make a set of high-level recommendations for what the European Commission should do to help strengthen the search and discovery ecosystem. But we also want to hear from the wider community about what topics we might not have covered, and further deepen our understanding of emerging issues in this space in the coming months.
If you want to take part in these conversations, do join the conversation on our NGI Exchange Platform.