When it comes to evaluating new technology, there’s a lot to consider.
The deeper you get into your evaluation, the more you’ll find it’s not just identifying priorities, looping in the appropriate stakeholders, and nailing down budget. You’ve got to uncover which tools and vendors may overpromise and underdeliver—and which can actually help solve the challenges your team and company face.
In her latest ebook, Krysta Johnson, Head of Legal Operations at Lexion, shared her top 10 tips for recognizing red flags in your search for AI-powered legal tech. From recognizing potential pitfalls to knowing which questions to ask, there’s a lot to unpack.
Here are three of Krysta’s top tips when evaluating new AI-powered legal technology (including CLMs).
Red Flag #1: The vendor doesn’t ask about your problems and requirements before showing you a 1-on-1 demo.
If a vendor isn’t trying to learn about your unique requirements before a 1-on-1 demo, that’s a red flag. I’ve personally found that vendors who don’t take the time to understand my specific needs and goals before pitching their technology are vendors who don’t have my best interests at heart.
More often than not, vendors who fail to take the time to learn your objectives before a 1-on-1 demo are relying on the novelty of AI to sell their product rather than demonstrating how their product’s AI capabilities align with your unique objectives.
Take it from Cassie Pless, VP of Sales at Lexion:
“One of the reasons that Lexion runs weekly, live demos on LinkedIn is because we want anyone interested in our AI-powered CLM tool to have access. That said, our 1-on-1 demos are where we take the time to map out exactly how our product meets the objectives and needs of a specific organization.”
Note: Before you start scheduling demos of AI tools, make sure you identify the pain points Legal is trying to solve and any stakeholder requirements. That way you’re prepared when the vendors you’re evaluating ask what those requirements are.
Red Flag #3: The vendor’s AI tool isn’t (easily) customizable.
I don’t know about you, but every legal job I’ve ever had has shown me that legal processes vary significantly from one organization to another. A tool that’s right for Company A may not be the tool that’s right for Company B.
An organization’s needs can also change over time. This is why it’s so important to be able to customize AI tools to match your language and processes (not the other way around). A lack of customization in a tool often limits its effectiveness.
To ensure you’re buying the right tool for your team, make sure you understand whether customization is self-serve or requires vendor support. If it’s self-serve customization, you’ll need to understand the level of technical expertise needed (and make sure you have resourcing to support it).
And remember: the latter can lead to delays that may have a real impact on usability and effectiveness.
Red Flag #9: The vendor isn't able to provide you with a clear plan for scaling usage.
As your organization grows, so should your AI solution. A lack of scalability of tools can lead to additional costs and disruption down the line when you’re rushed to find a more suitable vendor before the contract with your existing vendor expires.
Ensure that the AI tool you choose can handle increased data volumes and complexity as your legal operations expand (without blowing your legal tech budget out of the water). If your vendor can’t help you demonstrate clear, proactive strategies for scalability to meet your evolving needs without incurring prohibitive costs or operational disruptions, consider it a significant red flag.
A partner committed to your growth will offer flexible solutions that adapt to increasing data volumes, complexity, and user demands, ensuring your legal tech stack evolves alongside your organization.
There’s more where these came from
Want to fill in the gaps and dig deeper into all ten of Krysta’s tips for exposing red flags in your search for legal technology?
Download your free copy of Ten Red Flags to Look For When Vetting Legal AI Tools (Including CLMs) here.
