What Experience Designers Can Learn From Data Ethics

Clear, useful ethical standards for software engineers, data scientists, and designers are somewhat lacking.

While many existing guidelines gesture toward noble goals and seek to ensure work is being done in the public interest with transparency and respect for privacy, there remain significant blind spots. Questions of inclusion, principles for ensuring external consistency, and other issues affecting edge users either largely go ignored or do not offer clear guidelines on how to navigate such problems effectively.

At the same time, there is a growing body of literature being produced by designers, companies, and other institutions that begin codifying general principles and tips based on market research and industry learnings. Generally, these are shared in small trade publications, blogs, and mini-sites. Overall, there is notable overlap in the key goals they are trying to establish: things like transparency, equity, inclusion, and putting the human user first. However, these have not been institutionalized or formally distilled into a general set of professional principles. This is precisely why a new organization can play an important role in filling this gap.

Overview of existing guidelines

There are a few existing lists of principles and ethical standards for data scientists and software designers that are good points of reference. Chief among these are the ACM/IEEE-CS Software Engineering Code and the ACM Code of Ethics and Professional Conduct. These guidelines are published by professional associations and largely reflect an institutional perspective that looks at designers and data scientists as professionals in their field, rather than as people with tremendous influence over the shape of our world and how different people can engage with it. This professional perspective is a limited one.

There are other organizations doing similar work that is much closer in focus to the kind of approach we’re exploring in our research. One excellent example is Accenture’s universal principles of data ethics. These principles look at software and systems design from an ethical perspective by highlighting the importance of putting human concerns first, emphasizing the ethics of how the system engages the user at every stage, and ask designers to proactively consider negative downstream implications. Another key focus here is on governance and how it’s vital that these systems be auditable, transparent, and accountable at every stage – since it is likely that they are being built with implicit bias in the first place.

The other really key set of principles worth mentioning here is Deon, an ethics checklist for data scientists created by Driven Data (a company that organizes crowdsourcing solutions for data science issues). Deon has made the rounds across GitHub, Reddit, and other forums and hubs where software developers gather – underscoring the pent-up demand for usable, relevant and forward-thinking ethical principles. Again, we see emphasis on accountability and transparency at every stage, from data collection to storage, analysis, modeling, and deployment. And we continue to see guidelines around ensuring transparency with users, privacy, configuration options, opt-in and opt-out options, bias in both methodologies and data sets, and other blindspots related to inclusion.

Inclusive Design Principles is another excellent checklist primarily focused on ensuring that design is inclusive of people with disabilities. They also provide practical examples for each of the points on the list, which makes this a great tool to understand how exactly these principles can be enacted in different fields. Overall, these guidelines also continue to emphasize user choice, consistent experiences across different types of users, and proactive considerations of how edge users might engage with the products.

Finally, stepping away from the more institutional checklists, there are many bloggers and writers who have put together their own checklists for ethical design and are sharing best practices across small trade outlets, blogs, and forums. These can be articles in places like UX Planet or posts on blogs like Towards Data Science. These lists largely echo the points we find in other guidelines, but are typically more geared to specific applications (like UX design, or focusing on inclusion in software development).

Key themes in data ethics principles

Across existing guidelines, checklists, and principles for ethical data science, several key themes emerge. Below, we’ve highlighted a few of the themes most relevant to design automation, and those that are on the cutting-edge of developments in the space.

Inclusion

One common theme is the importance of inclusion in software design. In particular, there is a growing dichotomy between accessibility and inclusion: in short, accessibility refers to the outcome of a design that makes it usable, while inclusivity refers more to the process by which these systems are designed in the first place. While the two go hand in hand, the growing importance of inclusivity reflects an approach where designers are increasingly expected (or should be expected) to use methodologies that explicitly and proactively consider edge users in the design process. Julian Kilker, Associate Professor, Emerging Technologies at UNLV, puts it another way: “Design ethics, in short, needs to move beyond making technologies accessible to all people, to making all types of people accessible to designers.”

The primary emphasis on inclusive design remains focused on people with disabilities, but is increasingly incorporating considerations around people with language barriers or low tech literacy. Overall, new organizations must emerge to bring more traditionally overlooked communities into consideration and to act as a leader in developing standards and guidelines by which to design better systems to meet their needs. This can be done through direct consultations with experts or with advocates from affected populations, through user feedback, or even by simply imagining how people with different abilities might engage with a given situation. Theoretically, there could even be a useful checklist that asks designers to consider how different people might approach their product (i.e., “How would a hard of hearing person use this?”; “How would a blind person use this?”; “How would a person with limited English proficiency use this?”; “How would this translate along XYZ metrics for right-left script?”).

Customization vs. streamlining processes

Another key theme that emerges in the automation and UX space is the importance of balancing customization and choice with streamlining processes and maximizing efficiency. Naturally, systems want to offer as much choice and customization to users as possible. But knowing that complex systems can be overwhelming and have a steep learning curve, it is increasingly important for designers to begin with a clear sense of what their system aims to accomplish, so they can ensure the final product offers a reasonable range of configuration options without sacrificing speed or efficiency. That necessarily varies from project to project, but the principle stands. There is a pressing need to develop guidelines for thinking these types of systems through, particularly from a perspective that aims to promote external consistency with other tools.

Transparency, confidentiality, privacy

As one might expect, transparency, confidentiality, and privacy continue to be major topics in virtually every one of these existing sets of guidelines and principles. Generally, the common point is that systems should be designed to be transparent about how they collect, use, and share data; confidential in protecting sensitive information; and private in respecting user preferences and legal requirements. This includes clear communication about data practices, robust security measures, and user controls over their personal data.

Accountability and governance

Finally, accountability and governance are crucial themes. This involves establishing clear responsibilities for ethical data practices, implementing mechanisms for oversight and redress, and ensuring that systems are auditable and explainable. The goal is to create a framework where organizations and individuals are held responsible for the ethical implications of their data and AI systems, fostering trust and mitigating harm. This often requires interdisciplinary collaboration and ongoing adaptation to new technological developments and societal expectations.