Exploring standards for algorithmic transparency

Andrea Hackl, Laura Reed, Djordje Krivokapic, Vladan Joler The Ranking Digital Rights annual Corporate Accountability Index currently evaluates 22 internet, mobile, and telecommunications companies on their disclosed policies affecting users’ freedom of expression and privacy. Given the urgent public interest concerns related to algorithms, we want to explore how the Index can also set standards for corporate disclosure and accountability around the development and use of algorithms. In co-facilitating this workshop, RDR will leverage its expertise in developing indicators—which are broadly applicable standards that enable researchers to measure and compare companies’ disclosed policies affecting users’ digital rights—while SHARE Foundation will draw upon its experience researching and mapping the influence of Facebook’s algorithm. We will begin the workshop with a discussion of what we mean by algorithmic transparency and what researchers and civil society groups have suggested as best practices. We will map out the different ways that such company practices and disclosures could be evaluated and compared with one another, based on our experience developing the Index methodology. Participants will then be broken up into smaller groups to brainstorm key considerations and potential elements for inclusion in an indicator measuring algorithmic transparency that could be used by researchers and civil society to determine whether companies are developing and deploying algorithms in a responsible and accountable manner. Discussion topics may include (but are not limited to): What standards and good practices are we seeking to encourage? How should transparency be defined in the context of algorithms? What would meaningful transparency look like? What obstacles prevent us from evaluating algorithmic transparency? What is realistic to expect companies to disclose? RDR will use the recommendations and feedback from workshop participants to explore whether and how to incorporate evaluation of algorithmic transparency into its Index.