Ensuring Quality in Data & AI: A Comprehensive Approach to Quality in the Age of Data & AI [Testμ 2023]

5 Pillar Model for the Quality of Data and AI System

After gaining relevant experience in software testing and quality assurance, Bharath came up with his five-pillar model to define quality for AI systems. He explained each pillar in depth as follows:

Pillar 1: Data Quality

Data quality is super important for any system to work well. This means the data should be fresh, complete, accurate, and consistent. Bharath walked us through various aspects of data quality:

• Data Profiling: Profiling data involves analyzing its characteristics, which assists in measuring data quality. Tools like DQ and Great Expectations are recommended.
• Data Cleansing: Anonymizing data, avoiding exposing PII (Personally Identifiable Information), and ensuring data validity are crucial for maintaining quality.
• Tracking Data Lineage: Understanding data sources, transformations, and usage is vital. Metadata management tools are highlighted for maintaining data lineage.
• Tool Frameworks: Apache’s tools and other data quality tools are available for addressing these aspects. Some great tools can be used, like Deequ.

Pillar 2: Algorithm/Model Quality

Bharath explained that algorithms play a great role in determining the accuracy of these models, which further affects the predictions they generate.

• Model Validation: Ensuring the quality of AI algorithms or models involves collaboration with data scientists, ML engineers, and others.
• Bias Testing: Detecting and mitigating bias in AI models is important. Tools like IBM Fairness 360 aid in bias testing to ensure fairness.
• Interpretability and Explainability: Understanding the model’s working, changes over time, and implications on results is crucial for success.
• Data Quality Impact: The quality of training and testing data directly influences model performance. Poor data quality can lead to unexpected results.

Each of these pillars — model validation, bias detection, explainability, and data quality — maps directly to practical testing strategies. Our guide on testing AI applications covers how to implement these checks across the AI development lifecycle.

Pillar 3: Infrastructure Quality

The infrastructure quality can have a direct impact on the performance and availability of the system. Bharat further broke it down into different parts:

• Infrastructure Testing: Maintaining infrastructure quality involves thorough testing to ensure optimal performance of systems and servers.
• Configuration Testing: Ensuring configuration consistency to avoid dependency issues and performance unpredictability.
• Continuous Monitoring: Regularly tracking the quality of infrastructure over time and adapting to changes is essential.

Pillar 4: Ethical Compliances

While many people wonder ethical compliances won’t play a major role in determining quality, Bharath explained its importance in maintaining the trust of customers and stakeholders.

• Ethical Considerations: Addressing ethical concerns is crucial, considering the impact of systems on users, values, and potential biases.
• Guidelines and Guardrails: Setting ethical guidelines and guardrails, such as respecting privacy, client data usage, and defining acceptable uses of AI models.
• Risk Management and Audits: Employing third-party audits to assess compliance and risk management, ensuring adherence to ethical guidelines.
• Regulatory Compliance: Ensuring data security, privacy, and regulatory compliance to avoid exposing sensitive data or violating regulations.

With the rise of AI in testing, its crucial to stay competitive by upskilling or polishing your skillsets. The KaneAI Certification proves your hands-on AI testing skills and positions you as a future-ready, high-value QA professional.

Pillar 5: Data Governance

Last but not least, Bharath highlighted the significance of Data Governance when considering the quality of AI systems. He further explained the aspects as follows:

• Data Governance Importance: Data governance combines technical and functional aspects to ensure proper data management and usage.
• Service Level Objectives: Defining service level objectives and policies related to data privacy, security, and authorization.
• Stewardship: Appointing stewards for data sources to ensure accountability and proper usage guidance.
• Audits and Checklists: Employing audits and checklists to validate compliance with established policies.
• Data Cataloging and Access Management: Proper cataloging and access management tools enforce policy adherence and control data usage.

Impact of Quality on Roles

As Bharath explained data quality, he discussed a significant change in the roles within the tech landscape, particularly for developers, QAs, and BAs. He emphasized that the roles are no longer limited to traditional developers, QAs, and project managers. With the rise of data and AI, new roles have emerged, including data professionals like data scientists, data engineers, and data product managers.

Impact on Developer Roles

Bharath discussed that developers today go beyond traditional coding and work with data specialists to understand cutting-edge storage and processing techniques. They execute cross-functional testing, provide rationales behind data decisions, and guarantee data validity to build systems that align with the five quality pillars.

Impact on QA Roles

He discussed how QA roles have extended beyond functional tests, including cross-functional and data platform evaluations. Today, engineers detect bias, verify ethical usage, and validate data quality, ensuring compliance. Bridging technical and non-technical teams, they advocate a holistic quality approach.

Impact on Business Analyst Roles

While discussing the impact of quality on BA roles, Bharath mentioned that BAs are pivotal in grasping ethical data usage and meeting regulatory demands. They communicate user needs, collaborate with QAs for compliance, and shape systems adhering to the five pillars of quality.

As the session approached the end, Bharath concluded by discussing the future of AI and data. He suggests everyone concentrate on their areas of expertise, remain open to skill-set shifts, and foster a comprehensive understanding, including compliance, ethics, security, and privacy considerations to sustain in the digital landscape. He recommended everyone adapt to the new changes and remain well-equipped in this new era of technology.