Assessing Accuracy Appropriately
Machine learning takes the trouble of of choosing the right test set for benchmarking quantum chemical methods.
Benchmarking in computational chemistry has become an arduous task. Reliable computational thermochemistry requires properties and processes to be predictable within and accuracy of 1 kcal/mol or less; sometimes as low as 0.1-0.2 kcal/mol.
Benchmarking new and lower-order methods against higher-order methods (or experiment, where available) is the accepted way of establishing accuracy, but the selection of test cases (test sets) has been largely been at the discretion of researchers and subject to personal preferences. The systematic evaluation of all cases, which is rarely done, often reveals weaknesses of methods and it can be difficult to assess the importance of these shortcomings to a specific research topic.
Benchmarking methods is necessary to understand the consequences of choosing one method over another.
A large comprehensive database is being constructed for future release, benchmarking a wide range of methods against CMQMC and the current “gold standard” in quantum chemistry, CCSD(T). Rather than recommending test sets that involve up to a thousand single point calculations to test just one method (let alone compare several), this project seeks to use machine learning to identify smaller sub-sets of structures ideally suited to benchmark a given method, for a given problem. Using multivariate statistical techniques such as archetypal analysis (AA) and k-means clustering to find the prototypes of a data set, a smaller, more representative test set can be constructed that will have the same mean and variance as a comprehensive set to test the accuracy (and weaknesses) of a method at a fraction of the cost.
For more information, contact the Project Leader, Dr Deidre Cleland.