Devshi Mehrotra is a 2021 Forbes 30 under 30 lister, and co-founder of JusticeText, a start-up using artificial intelligence to transcribe and analyse audio-visual evidence, including bodycam footage, interrogation videos, and more. By doing so, Devshi is helping public defenders be more productive and effective in their jobs, who serve a crucial role in the criminal justice system.
And Devshi is only just getting started!
Graduating in 2020, Devshi has been busy for several years, gradually combining her passions for computer science and criminal justice reform alongside her studies, and ultimately building a start-up that harnesses both.
In this interview we explore how combining computer scientists with lawyers can unlock new value and widen access to justice. We also discuss the landscape for women in tech and ideas on how the tech community and educational systems can do more to widen access and drive engagement across under-represented groups.
Unlike our career profiles to date, featuring individuals a decade or more into their careers, you are just getting started. But you’ve already done an incredible amount of exciting things. Can you walk us through your journey to date?
I’m a recent graduate. For my undergrad I studied at the University of Chicago, obtaining both my bachelor’s and master’s in computer science there.
After attending the University of Chicago I spent a year at Tsinghua University in China, where I received a masters in Global Affairs.
How was studying in China?
It was really exciting, albeit a little disrupted by the outbreak of COVID-19 whilst I was there!
The program was very different to my previous computer science studies, but it was incredibly eye-opening in terms of content, teaching and the variety of students from all over the world, including from India, Pakistan, Southeast Asia and all across Africa.
I really loved living and studying in China. I think many Americans still internalize this idea that the start-of-the-art is what is produced within our borders… but going to China, there is no question in my mind – China is so far ahead in many areas, whether its developments in public infrastructure, smart cities, transportation systems, and generally integrated approaches to managing a lot of urban challenges. In those areas, China is really pushing ahead hard and fast as compared to the USA.
I should add that I studied Chinese for four or five years before studying abroad in China, and so it was also a culture in which I had personally invested a lot of time and energy getting to know. So from that perspective, it was great to immerse myself in the culture first hand.
And between your various studies you’ve managed to cram in a lot of exciting internships. Can you tell us some more about those experiences?
Yes, alongside my studies I completed internships at several big tech companies, predominantly in their machine learning research groups.
In my first summer I interned for Facebook, collaborating with two fellow interns to build Reach, a mobile platform designed to connect volunteers with service opportunities in their community. As part of that internship I personally presented our project to Mark Zuckerberg as recognition for having built the Best iOS Application in the Facebook University program.
Into my next summer I got to work at Google Brain, working on a variety of machine learning and robotics projects. That internship got me really excited about the world of AI research and led me to spend time in London at DeepMind.
You are also very passionate about social justice reform, a very different domain to computer science, albeit one that impacts everyone. How did this interest develop?
While my undergrad major was highly technical, I also spent a lot of time in the city of Chicago, getting involved with local organizing efforts around criminal justice reform.
Unsurprisingly given the events of 2020, the national conversation about criminal justice reform has really exploded over the last year.
But in many urban communities, criminal justice reform has been an on-going conversation for decades. Policing and community relations are often fraught because of America’s long history of racial discrimination.
So on the one hand I was spending a lot of time involved in these really incredible activism efforts, but on the other hand, spending summers working in high tech environments and communities. Two very different worlds.
The juxtaposition of these two experiences lingered with me for some time. In particular, the disparity of resources between the two environments stayed with me. It also made me realise that what I really find purposeful and meaningful was engaging in direct action.
And is this why you started JusticeText?
Yes, JusticeText is how I’ve melded my interests in justice reform and my skillset as a technologist.
I should say however, that initially I had no idea what that was going to look like, but one day I just decided, “you know what, I’m going to show up at my local public defender’s office, start a conversation with them and see where I can take it.”
And that’s how JusticeText got started. It started just before going to China, while I was still an undergrad in Chicago. It began as a school side project, just something that me and Leslie Jones-Dove, my co-founder, whom I’ve known since our first day as freshmen, were working on.
We built a rather bare bones prototype, but it was a start!
After that we began outreach to different attorneys, agencies and so on. I continued this alongside Leslie whilst I was in China, often meaning I was having to jump on calls at 11:30pm local time in China to do product research, interviews, set-up trials and move pilots forward with existing and potential users. During that time Leslie took on a lot of development work.
I actually didn’t expect to commit to building JusticeText full time until after graduating, but I am so glad I made that decision.
Can you tell us some more about JusticeText?
A major problem public defenders face is high volumes of data. This problem is only getting bigger. Whenever an event happens that has some police, legal or other implication there is a ton of data created via, and collected by, the US justice system, from body camera footage, interrogation audiovisual data, transcripts, jail phone calls, witness footage from smart devices and so on.
This is perhaps not all that surprising when you think about modern life and how much technology permeates everything we do.
The surprising part however is that there is very little to no infrastructure in place for public defenders to make sense of this information quickly, effectively and thoroughly. Instead, many public defender attorneys themselves or with the help of others have to manually transcribe, organise and sift through hours and hours of data, in particular an increasing body of audiovisual data, trying to synthesize information and understand the full facts of a case in order to ensure justice, and access to it.
Layered onto that, these public defenders have huge caseloads: 200 maybe 300 cases at any given time.
Even the best attorney will struggle with this heavy workload and voluminous data. The risk is that something gets missed, evidence perhaps that could in fact be decisive to a case, whether advantageous or harmful to a client. There’s a lot at stake.
As technologists, we saw inefficiencies that were contributing to injustice. We felt technology solutions could help solve some of these problems.
So the tool itself, JusticeText, is a web platform. Public defenders upload their discovery data directly to the system.
Attorneys can then quickly confirm things such as the primary language spoken, tag by evidence type, add metadata as is relevant.
Any audiovisual data is also transcribed so that every single word has a time stamp. And so you can click on that word, and jump to that part of the video. We have also started doing some preliminary keyword extraction of terms that might be relevant to legal analysis.
That’s very cool. With regard to natural language processing in this context, is it possible to apply something like sentiment analysis to provide leading indicators for attorneys, e.g. highlighting potentially hostile language, or vice versa?
That’s one of the areas where we would love to move forward! One of the things that I’ve noticed is that sometimes it’s as much the intonation of the voice as it is the raw transcription that indicates whether or not someone was angry or upset.
And so the experiments we’ve run have been inconclusive in the sense the insights we can provide are interesting and clever, but perhaps not the most useful. At least not yet!
It’s an area with which we will keep experimenting, and we’ve explored numerous off the shelf tools in that regard as well. There’s undoubtedly lots of ways we can gradually increase the sophistication of the technology beyond the automation of searchable, tagged and taggable transcripts.
Just to back up for a moment, you mentioned an initial idea before the current JusticeText concept – what happened with that idea?
Of course, the first idea already existed as we discovered quite quickly.
As a result, we flipped the conversation, explained our background, the skillset that we bring and asked open questions, “are there any things that you’ve been struggling with that you think could be helpful if solved?”.
The interesting thing about being a technologist in the legal space is that you don’t bring domain expertise, so the number one thing that you can do, is bring empathy and a willingness to listen and to learn about problems. You can also help by looking at problems in a different way to those deep in a particular domain.
It’s worth saying that I was not tied to our original idea at all. It was a starting point for a larger discovery process. If several people are telling me that something else provides value, then we might as well focus our time and attention on solving that problem. If they are saying the opposite then it’s a sign there are better problems to identify and solve.
That’s a great point. I agree that outsiders often breathe fresh ideas into other domains. If you come to domain A from domain B, you aren’t path dependent or predisposed to accept the status quo but more likely to question why something is a certain way. That can be really hard to do if you’re deep inside your own domain.
So how did you take things forward from these initial problem identification discussions?
We’ve been in product development mode for the past year, and after I graduated in June of last year, I gave myself two months to try and raise the initial funding for JusticeText.
My thinking was this: if I can succeed at some initial fundraising, then I’ll commit full-time. And thankfully we raised a pre-seed round of around $300,000.
That fundraising has helped us add two additional engineers – it’s made a world of a difference for us.
Our extensive efforts running user interviews, feedback sessions, surveys, and so on is paying off. We’ve moved on from scoring 3s to 6s out of 10 to consistently scoring 9s and 10s out of 10. It’s a long process, it’s not easy, but totally worth it! And we’re still only in the very early iterations of the product, so the long term view is very promising.
Now we’re hitting our stride in terms of product, my aim is to shift from day-to-day product management into launch readiness. We are hoping to officially launch in summer of 2021. So far we’ve had very encouraging conversations with various states and counties with regard to winning government contracts so that our platform ends up in the hands of more public defenders.
What was it that improved the user feedback scores? Was it new features, or a reframing of existing value?
There’s a couple of things.
One is the messaging. Initially, attorneys struggled to see the big picture, the problem we were trying to solve. When they heard terms like “transcript” and “transcribe” they tended to see it as simply transcription, which they already do, albeit very slowly. Framed that way, they couldn’t differentiate the value vs. the status quo.
But in reality, the value isn’t just the transcription technology, but the workflow we create around that technology. That workflow is JusticeText’s real value: making it easy not only to transcribe, but also to review, edit, tag, collaborate around, and generally leverage transcription data in ways that make sense to attorneys. We’ve managed to improve product and the user experience, and the surrounding messaging, so that this value is more immediately obvious to users.
The other thing is the accuracy question.
Like most machine learning tools, it’s not always terminal that the system isn’t 100% accurate. Initially early users used to say “hey, it’s only 75% or 80% accurate, I can’t use this etc”. So we had to do a little reframing to help users understand they needn’t take an all or nothing approach – even less than 100% accuracy is still a massive time save and value add vs. the status quo processes, especially when you layer on the workflow features.
Cognisant of this feedback, we’ve been more intentional about addressing those issues in the last couple of months, which has made a big difference.
That’s a really interesting observation regarding AI accuracy. We’ve seen a lot of AI tools fail, or underwhelm, not because of accuracy but because of poor or entirely absent workflow tools. Attorneys tend to get overly focused on accuracy, because in their mind it speaks to risk, which is what they know. What they are sometimes less good at is understanding how other factors have outsize impact, in particular the power of good workflows when paired with “accurate enough” AI. There’s a reason other knowledge based or highly skilled industries – such as surgeons, pilots and astronauts – obsess over workflows, but lawyers don’t always appreciate their power.
On this point, do you think having a computer science background helps you see these sorts of legal challenges differently?
The time that I’ve spent with the Silicon Valley community has been very influential. They are laser focused on workflow tools, even for what seem like the tiniest things.
VC funds are similar. They seem to try and automate as much as they can, whether that’s reviewing emails, pitches and organising calendars etc.
The individuals and teams in those spaces just seem to have this immediate urge to innovate problems away, especially anything remotely capable of being turned into a workflow of some kind. And everyone in that space is super interested in what tools and methodologies each other are using to be productive. It’s embedded within that environment; it’s the norm not the exception.
This background definitely helps me see opportunities for better workflows when I meet with attorneys to discuss their pain points and what they do. There is a huge opportunity to improve things, and a growing number of young technologists who are keen to help innovate this sector because they want to make a difference.
Have you seen openness to, and enthusiasm for, this type of thinking when working with lawyers?
It depends. Without wanting to generalize, the incoming generation of public defenders seem more open to these ideas and ways of working. They’ve been the easiest adopters so far. Perhaps that is because they are as yet less ingrained and path dependent on the traditional ways of working.
But I do think, in part at least, this incoming generation of lawyers have different expectations because when they’re dealing with their finances or social media or transportation, they expect a high-quality user experience. I think the lack of these sorts of experiences in legal processes is probably at odds with those expectations, which is perhaps a good thing and will encourage the incoming generations to be change agents and prioritise workflows to aid their own work and enhance client service.
You’re right – there’s definitely a widening gulf between how consumers experience tech and how they experience tech in legal products and services. There’s not too many good reasons why this gap persists!
Thinking again about you as a founder, can you walk us through your interest in start-ups, and what tips you would suggest for starting a business?
I went through my phase as a college student where I just wanted to start a business for the sake of starting a business. But I quickly realized that the only things worth pursuing as an entrepreneur were causes I was genuinely passionate about.
In fact, JusticeText absolutely didn’t arise from these earlier interests in start-ups.
Quite the opposite. Rather, JusticeText was sprung from my interest in justice reform, which became a bit of an obsession. I couldn’t stop thinking about the underlying issues, day in, day out. I was taking classes, attending lectures and just trying to learn as much as I could. I would say to any student entrepreneur, or any entrepreneur generally, to look for issues and problems that you and others really care about solving. College is such an incredible environment from which to explore these challenges and opportunities. You are surrounded by so many brilliant minds and have such incredible access to them.
I’d also say that being a student opens a surprising number of doors. People tend to give you the benefit of the doubt and want to help you because you’re a student. That means there’s a world of opportunities to seek out interesting people and problems.
If as a student, you can leverage these opportunities once you find that problem that you can’t get out of your mind, that you’re thinking of day in and day out, you’re probably onto something. It might not be the thing, but it is more likely to be a goer than setting out with the vapid objective of just starting a startup for the sake of it. At the very least, it will be the starting point for a productive set of discussions that may lead to new insights you can action and solve via a start-up business idea.
So don’t sit in your dorm room worrying about market mapping everything and coming up with your financial models in a vacuum. Get out there! Literally go to the community that you care about where the problem is experienced the most. Be curious, ask questions. Do work for free in the beginning. Volunteer your talents and experience. Exchange experiences – help someone, and they will help you.
That might mean helping someone solve a part of the problem, who later becomes an early adopter and champion of your mission. If you can find individuals who are frustrated by the issues you’re seeking to solve, they will give you a lot of time. People like to talk about themselves and share their problems, especially if they think you can help solve them, and if you are genuinely interested in solving them.
That’s your first pilot user, and perhaps a first customer.
Once you lock in those initial interactions, lean into those relationships. Really invest in them. We did this for six months, getting immersed in the users, their jobs to be done and their pain points and the wider structures of government and government technology. I’m more a govtech expert than a legaltech expert as a result.
As a result, we got to know a lot of the systemic barriers of government processes, including procurement and so on.
To find these interactions, you have to be proactive: shoot your shot.
Sure a pre-existing network helps, but don’t think that you need one to do this type of work, or that if you don’t have one you can’t start.
As long as you’re genuine and authentic about the issues and the people, you will easily build your own network.
That’s really good advice. Thank you for sharing. It’s great to see you and Leslie diving straight in, meeting with real users and really getting to know the pain points and other structural drivers behind the status quo.
Changing directions, I know you’re passionate about widening access to computer science, in particular your CompileHer initiative – great name by the way – to build role models and encourage women into tech. Do you mind sharing your thoughts on this subject?
In the USA, one of the interesting things that you’ll notice, is that a lot of the women in tech come from immigrant families.
For whatever reason, we tend to come from a family background that really encourages and supports young girls. Now that sometimes means there are unreasonably high expectations placed on young women, especially when it comes to succeeding in STEM subjects like maths and science etc.
Ever since I was little, that expectation was built in, that I would pursue STEM interests and that they were objectively valuable skills to hone.
But it wasn’t an expectation I saw shared in any of the TV or other media I or my friends encountered. It was an expectation stemming from my parents.
What I thought was normal – prioritisation of STEM excellence and similar – I soon realised wasn’t the experience of many others.
Many women don’t have parents or families with that inbuilt understanding of what it means, and what is required, to succeed in STEM subjects or how rewarding those subjects can be in every sense. There aren’t any STEM heroes, or glamourized individuals aimed at young girls. There’s plenty of male role models, but few females.
I wanted to change that. I just knew I had to do something. And CompileHer was one of my biggest passions in college. It’s an organisation I led an undergraduate, spending hours every week educating young girls about computer science and engineering, role models and just trying to get girls aware of, and enthusiastic about, STEM subjects.
I really wanted to plug the gap between different communities.
I wanted to do everything I could to ensure that the girls in my school have every single opportunity that any other student might have.
It was fantastic building these relationships as a college student with teachers, principals, parents and young girls. Occasionally I hear stories of how some of the girls have gone on to pursue interests in STEM subjects, which is really meaningful to me!
In terms of what we did, we organized hackathons, conferences, workshops and things like that.
That message I conveyed wasn’t so much simply “everyone should learn computer science” but more specifically “you should learn computer science because there’s challenges that you uniquely understand via your lived experiences, that aren’t shared by everyone and without you those challenges won’t get solved”.
What do you think the best way is to encourage more women into tech? Do you think it’s these grassroots, early interventions to build awareness that these opportunities exist and that women have unique experiences they can bring to a traditionally male dominant domain?
It does help. In some cities across the US, school districts are actually mandating computer science education as a high school graduation requirement. So there is now this system wide change being enacted that is going to make a world of difference, it’s called Computer Science for All. They also have specific focus on getting under represented groups into tech.
It’s fantastic, and the result of numerous incredible people devoting years and years towards building curriculum and advocating for change.
So that’s one part of the equation – making computer science a pillar of education generally.
The other part is tackling the fact that the majority of folk in tech aren’t women. This can be incredibly discouraging as a woman if you look around and there’s very few women in general and in senior roles. And I felt that way many times.
In tech mentorship is also an issue. In tech you are encouraged to try things, crash and burn and hopefully learn from the results. But for a lot of women, they feel hesitant to crash and burn before learning all of these things first. Creating environments where women engineers can come together, e.g. young girls who are trying to get into the space, can make a huge difference as well.
Do you have any books, podcasts or other media you’ve found influential with regard to what we’ve discussed?
Yes, a major reason for starting JusticeText was because of The New Jim Crow by Michelle Alexander. This book is now a very well known book. Professor Alexander is a lawyer and activist and her book explores the systemic biases of the American criminal justice system and how it functions as a contemporary system of racial control, relegating millions to a permanent second-class status even as it formally adheres to the principle of color blindness. A lot of the time when people think about racism, they think it’s limited to discriminatory comments or actions that are overt, or individual bad actors making bad decisions. This book takes a systems level approach, which tells a different story of the many hidden – but powerful – structures that contribute to racial injustice and require reform.
Interested in legaltech careers like Devshi’s? 
See here for our legaltech careers guide for advice and tips on the roles, organisations and routes into legaltech.
The post Legaltech Careers: Devshi Mehrotra, CEO & Co-Founder of JusticeText appeared first on lawtomated.