Ethical guardrails needed to make generic AI work for healthcare

Railing is a hot topic in the world of generative AI. Without much government regulation, the onus is falling on businesses and leaders to create a safe environment for users to interact with generative AI.

One of the biggest problems reported to date is data “hallucinations” or inaccuracies in the data. These key issues become apparent when users without sufficient experience rely heavily on generative AI tools.

Edwin Pahak, vice president of development at AI-powered services company Quant, runs his organization with a “clean data” mindset and believes that generative AI needs to prioritize source collaboration as much as it generates it. to the correctness of the answers. He argues that a two-lane understanding of the right answer and data usage is the foundation of a successful generative AI platform.

We interviewed Pahak to talk about his focus on data governance, AI tool ethics, guardrails, and data validity.

Q. Please talk about your focus on the clean data mindset.

One. Algorithms are only as reliable as the data they feed on. By fostering strong relationships with data sources and ensuring transparent data collection practices, AI modelers can develop mutually beneficial collaborations that promote accurate and unbiased AI outputs.

The collaboration of multiple data sources empowers AI systems to gain insights from different perspectives, preventing biases and skewed representation. Clean data is also about creating new, synthetic data from expert insights so that you can significantly increase data quality by providing additional information and filling in gaps in existing datasets.

Q. Let’s discuss AI tool ethics. You say that AI tools need to be trained to collect data only from sources that have agreed to it. How can companies do this?

One. To train AI models with data only from consented sources, the primary approach involves implementing rigorous data collection practices and obtaining explicit consent. This process generally includes the following steps:

• First, clearly state the purpose and scope of data collection, provide transparency about how the data will be used.

• Second, seek informed consent from individuals or organizations, ensuring that they understand the implications and are willing to share their data for specified purposes.

• Third, establish robust data management protocols to ensure that data is managed securely and in compliance with privacy regulations.

• Fourth, regularly review and update consent agreements, so that individuals can withdraw their consent if they wish. By following these practices, AI models can be trained exclusively using data obtained from sources that have provided explicit consent, ensuring ethical and legal compliance in the data gathering process.

Q. Railing is a big topic in AI. How can organizations install these guardrails without hindering the effectiveness of AI tools or keeping employees tied to security processes?

One. To install guardrails into AI models without hindering effectiveness or putting too much pressure on employees, organizations must design AI systems from the ground up that are inherently attuned to human values ​​and exhibit safe and ethical behavior. Achieving a fully self-governing AI system is a complex task but it is possible.

The most important aspect of this is to thoroughly test the data integrity. This includes ensuring the accuracy, completeness and consistency of the data used in the deployment. By rigorously addressing data integrity through QA processes, you are ensuring that models are built on accurate and consistent data, leading to more reliable predictions and insights for end users.

Q. Data validity is another key in AI. Please explain how using verified and valid data can help mitigate some of the risks associated with generative AI tools.

One. Using verified and valid data can help mitigate the risks associated with generative AI tools by ensuring the accuracy and reliability of the outputs.

By leveraging trusted data sources, organizations can reduce the likelihood of biased, misleading or harmful content being generated, thereby increasing the overall integrity and security of AI-generated outputs.

Q. With all of this in mind, what should CIOs and other health IT leaders in healthcare provider organizations be focusing on as generic AI tools continue to grow in importance?

One. There are several key areas CIOs and other health IT leaders should focus on whether they intend to build an AI system in-house or partner with a vendor.

Firstly, they need to ensure robust data governance practices to maintain the quality, privacy and security of the data used by these tools. Additionally, they should prioritize ethical considerations such as transparency, explainability, and objectivity to build trust in AI outputs.

Furthermore, fostering collaboration among all stakeholders – physicians, data scientists and IT teams – is critical to effectively integrating generative AI tools into clinical workflows while aligning with regulatory requirements and patient safety standards.