Safe and Responsible AI Engineering

 

Science and Engineering-Driven

 

Industry and Impact-Focused

Selected Publications:

LLM Agent Architecture Design

• Qinghua Lu, Liming Zhu, Xiwei Xu, Zhenchang Xing, Stefan Harrer, and Jon Whittle. “Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agents.” arXiv preprint arXiv:2311.13148 (2023).
• Liu, Yue, Sin Kit Lo, Qinghua Lu, Liming Zhu, Dehai Zhao, Xiwei Xu, Stefan Harrer, and Jon Whittle. “Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agents.” arXiv preprint arXiv:2405.10467 (2024).
• Jingwen Zhou, Qinghua Lu, Jieshan Chen, Liming Zhu, Xiwei Xu, Zhenchang Xing, Stefan Harrer. “A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Model.” arXiv preprint arXiv:2408.02920 (2024).
• Qinghua Lu, Liming Zhu, Xiwei Xu, Zhenchang Xing, and Jon Whittle. “Towards Responsible AI in the Era of Generative AI: A Reference Architecture for Designing Foundation Model based Systems.” arXiv preprint arXiv:2304.11090 (2023).
• Qinghua Lu, Liming Zhu, Xiwei Xu, Yue Liu, Zhenchang Xing, and Jon Whittle. “A Taxonomy of Foundation Model based Systems through the Lens of Software Architecture.” arXiv preprint arXiv: arXiv:2305.05352 (2023).
• Qinghua Lu, Dehai Zhao, Yue Liu, Hao Zhang, Liming Zhu, Xiwei Xu, Angela Shi, and Tristan Tan. “Evaluating the Architecture of Large Language Model based Agents.” (2024).

Swiss Cheese Model for AI Safety

• Md Shamsujjoha, Qinghua Lu, Dehai Zhao, and Liming Zhu. “Designing Multi-layered Runtime Guardrails for Foundation Model Based Agents: Swiss Cheese Model for AI Safety by Design.” arXiv preprint arXiv:2408.02205 (2024).

AI/Agent Safety Evaluation

• Boming Xia, Qinghua Lu, Liming Zhu, Zhenchang Xing, Dehai Zhao, and Hao Zhang. “An Evaluation-Driven Approach to Designing LLM Agents: Process and Architecture.” arXiv preprint arXiv:2411.13768 (2024).
• Boming Xia, Qinghua Lu, Liming Zhu, and Zhenchang Xing. Towards AI Safety: A Taxonomy for AI System Evaluation. arXiv preprint arXiv: 2404.05388 (2024).

 AgentOps

• Liming Dong, Qinghua Lu, and Liming Zhu. “A Taxonomy of AgentOps for Enabling Observability of Foundation Model based Agents.” arXiv preprint arXiv:2411.05285 (2024).

AI Risk Assessment and Mitigation

• Sung Une Lee, Harsha Perera, Boming Xia, Yue Liu, Qinghua Lu, Liming Zhu, Olivier Salvado, and Jon Whittle. “QB4AIRA: A Question Bank for AI Risk Assessment.” arXiv preprint arXiv:2305.09300 (2023).
• Sung Une Lee, Harsha Perera, Yue Liu, Boming Xia, Qinghua Lu, and Liming Zhu. “Integrating ESG and AI: A Comprehensive Responsible AI Assessment Framework.” arXiv preprint arXiv:2408.00965 (2024).
• Sung Une Lee, Harsha Perera, Yue Liu, Boming Xia, Qinghua Lu, and Liming Zhu. “Responsible AI Question Bank: A Comprehensive Tool for AI Risk Assessment.” arXiv preprint arXiv:2408.11820 (2024).
• Boming Xia, Qinghua Lu, Liming Zhu, Sung Une Lee, Yue Liu, and Zhenchang Xing. “Towards a Responsible AI Metrics Catalogue: A Collection of Metrics for AI Accountability.” In 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN ’24) (2024).
• Qinghua Lu, Liming Zhu, Xiwei Xu, Jon Whittle, Didar Zowghi, and Aurelie Jacquet. “Responsible AI pattern catalogue: A collection of best practices for AI governance and engineering.” ACM Computing Surveys 56, no. 7 (2024): 1-35.

Frameworks/Tools:

• ESG-AI Investor Framework – Assess RAI Practices upon Existing ESG Foundations
• Responsible AI Pattern Catalogue – Over 60 Best practices for Operationalizing Responsible AI from a System Perspective
• Agent Design Pattern Catalogue – 18 Design Patterns for Designing Foundation Model-based Agents
• Responsible AI Chatbot – Automatically Assessing AI Risks
• AIBOM (AI Bill of Materials) Generator – Comprehensive Record that Supports Accountability throughout the AI System’s Lifecycle
• VulBOM (Vulnerability Bill of Materials) Generator – Specifying Vulnerable File & Function, Vulnerability Propagation Path on the top of Software Bill of Materials
• AI Discovery Tool – AI Capability Detection in Android Apps
• Responsible AI Question Bank – Holistic AI Risk Assessment Toolkit
• Sapper No/Low Code AI Engineering Platform
  • Yu Cheng, Jieshan Chen, Qing Huang, Zhenchang Xing, Xiwei Xu, and Qinghua Lu. “Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chains.” ACM Transactions on Software Engineering and Methodology (2023).
• Guardrails Services/SDK for LLM Agents – Realtime Assessment and Protection for Safe LLM Interaction
• UI Guard
  • Jieshan Chen, Jiamou Sun, Sidong Feng, Zhenchang Xing, Qinghua Lu, Xiwei Xu, and Chunyang Chen. “Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile Applications.” In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pp. 1-20. 2023.
• Context-Aware Workflow Automator
  • Universal Task Assistant – A Universal Smart Device Assistant
• Interactive UI Prototype Generator – Controllable & Explainable GUI Prototype Generation Process Aligning with User Intentions
• Guidelines-Based Frontend Code Repair – Frontend Code Analysis and Repair based on Design Guidelines & Rendered Page
• AgentOps platform – Automatically Tracking Agent Artefacts at Runtime
• Agentic Compliance Guard
  • Automatically checking the agents’ workflow log data
  • Van Beest, Nick, Heerko Groefsema, Adrian Cryer, Guido Governatori, Silvano Colombo Tosatto, and Hannah Burke. “Cross-instance regulatory compliance checking of business process event logs.” IEEE Transactions on Software Engineering (2023).

New and Upcoming Book

Contact

Qinghua Lu: Qinghua.Lu@data61.csiro.au