Skip to Content
Get clarity on how to become AI-ready. Take the Assessment

The ITX Checklist for Evaluating AI Tools (Step 1)

Step 1: Assess competing stakeholder needs to identify the right tool for the job

This blog is the first in a series that explores ITX’s approach to AI tools integration – our proven method for vetting, selecting, and deploying artificial intelligence into our workflows. See Step 2 after reading Step 1.

Software product development companies face unrelenting pressure to stay ahead of tech’s accelerating pace. This is not a new phenomenon, of course. But when paired with an already volatile market environment and its ever-lower barriers to entry, just keeping pace becomes more than many companies can manage. Add artificial intelligence into the mix, and the strain can push them to the breaking point.

In the “earlier” days of AI (let’s face it: AI is still just a baby), every new tool felt like ‘a hammer in search of a nail.’ It felt like there was no job AI couldn’t do. These days, though, the calculus is changing. Tech companies that have been around for a while have figured out that specific stakeholders need specific tools with specific applications. And it takes a skilled craftsperson to choose the right tool for the right situation.

Evaluating Stakeholder Concerns

ITX’s deliberate, policy-driven process for the selection and use of new tools and technologies is largely the same as it’s been for years. Some might consider this way of working out of touch given today’s pace of rapid change; we believe our approach to vetting AI tools demonstrates a forward-thinking framework that delivers workability: a condition that is optimally suited for generating favorable results; the capacity to be highly effective.

Check out ITX’s Artificial Intelligence Hub to learn more about the underlying principles and rules that govern our approach to AI adoption and integration.

A Hierarchy of Constraints: Balancing Organizational, Technical, and Team Needs

Beginning with client consultation, our teams present and describe the concept of AI tool application generally – including exposure to risk – and only then explain in detail the opportunities for improved quality and efficiency.

Some clients have internal, legal, or regulatory constraints that restrict their use of AI. Their input is crucial both at the start and throughout the process.

Kevin Dolohanty
Director of Infrastructure and Security, ITX

Some clients prohibit the use of AI tools within their ecosystem; our tools policies ensure we respect their wishes and firmly establish their needs before our needs as product teams and individuals. Others have internal, legal, or regulatory constraints that restrict their use of AI. Their input is crucial both at the start and throughout the process. ITX teams evaluate each new tool and technology based on the following concerns:

Organizational.

Client Direction.

  • Required Use. Does our client require use of a specific AI tool? A client may have an AI system already in place that requires us to operate within pre-existing boundaries.
  • Constrained Use. Is the client’s use of AI limited by internal, legal, or regulatory constraints? Client input is crucial throughout the process, but especially at the start.

Legal.

  • Restrictions. What restrictions are in place that might harm our clients or ITX? By policy, ITX reviews all software licenses for subsequent approval or rejection.
  • Protections. Does the tool’s provider offer protections against claims related to the use of outputs generated by the AI tool? Having achieved Silver Business Partner status with IBM, ITX is able to invoke the standard contractual IP protections for IBM products through watsonx™. In addition, ITX leverages Microsoft™ Copilot which, like IBM, is committed to indemnifying companies against IP infringement claims related to the use of outputs produced by Copilot.

Value.

  • For the cost of the license, does the tool deliver sufficient value? Reduce waste? Drive efficiency? ITX is in the business of delivering business value; if we already have a tool that is in place and well established, we think long and hard before subverting or bypassing it with a new, unvetted tool recommended by a team or individual.

Technical.

Fitness. Does the tool or technology perform the job it’s been hired to do? This is a question asked within the context of both quantifiable and intangible factors.


Maturity. Is the tool or system out of beta – i.e., operational and well established? ITX does not use its clients or team members as quality assurance for AI tool providers.


Security. Does use of the tool introduce risk? Any tool or technology we introduce into our workflows must protect the integrity of our and our clients’ data.


Compatibility. Does the new tool or technology work well with existing systems?


Lifecycle. If we introduce a new tool or technology, what is done with the former tool’s data? Adoption of any new system should consider that system’s full lifecycle, maintenance, and eventual retirement.

Team / Individual.

Specific need. Does a team or team member have a problem unique to them? The purpose of some AI tools is narrow, creating a need that is not required at the organizational level.


Individual preference. Personal preference for an AI tool must not conflict with organizational, technical, or team considerations. Even if a team member has discovered an AI tool that increases their efficiency, the tool needs to be evaluated against superseding concerns.

Conclusion: Balance Stakeholder Needs to Identify the Right AI Tool

Like our approach to all new technologies, ITX’s examination of AI tools is both deliberate and meticulous. By considering and weighing the needs of all stakeholders, we are able to maintain high standards of security, scalability, and compatibility.

We prioritize client input and legal/ethical considerations, assess the maturity and fitness of the tools for the job to be done, and weigh their value and potential to drive efficiency. This ensures that every tool we adopt enhances our workflows without introducing undue risk.

Subsequent posts in this series will delve more deeply into ITX’s process for selecting and integrating AI tools into our product development workflows. Learn more by checking out ITX’s Artificial Intelligence Hub, which explains the underlying principles and rules that govern our approach to AI adoption and integration.


Peter Sullivan's portrait Picture

Peter Sullivan is Producer of ITX’s Product Momentum podcast and a student of Product and Design processes that work. As ITX’s Marketing Content Lead, he spearheads our efforts to deliver thought leadership that helps Product makers and UX designers understand and shape the future. 

Like what you see? Let’s talk now.

Reach Out