Towards a standardization framework for benchmarking and performance assessment in the AECOO community

Erik Poirier, PhD, M.Eng, B.Sc.Arch.,LEED AP

buildingSMART International is investigating the possibility of developing a standardized performance assessment framework for the global AECOO community. This initiative aligns with buildingSMART Canada’s efforts to develop such a framework for the Canadian AECOO community. This article summarizes the key elements that were presented and discussed at the bSI summit in Paris in March 2018 and proposes a series of next steps for Canada’s involvement at the international level and what it means for Canada moving forward.

Released in January 2015, the Roadmap for Lifecycle BIM in the Canadian AECOO Community identified evaluation and assessment as a key activity in the deployment of BIM on a national scale. The Statement of Intent that accompanied the roadmap clearly indicated that one of the six key principles for the effective deployment of BIM was the continuous monitoring and evaluation of the transformation process. Indeed, performance assessment and benchmarking has received significant attention in certain countries as a key activity for the AECOO community (see CII’s benchmarking effort in the US or Constructing Excellence’s KPI Zone in the UK). Alas, while efforts have been deployed to mirror such initiatives in Canada (for example the COAA’s initiative, research projects in various universities or the buildforce initiative (aimed towards contractors)), there lacks a recognized national (or even provincial) benchmarking and performance assessment framework that can consistently support a wide breadth and depth of evaluation and assessment that includes: industry, organization, project and individual performance, capability and maturity, as well as deployment. These international and national initiatives and research works have produced a vast array of key performance indicators, metrics and measures, yet these measures and the approaches through which they are operationalized vary which hinders data collection and analysis which limits interpretation. There is therefore a need to develop a standardized framework for benchmarking and performance assessment in the AECOO community.

In this context, and in its role as an international standardization body, buildingSMART International has launched an initiative to support the development of a standardized performance assessment framework which addresses this internationally recognized need. An inaugural session to present and discuss this initiative was held at the bSI summit in Paris in March 2018. The session was chaired by Richard Petrie and Patrick McLeamy, and attended by a number of delegates from around the world including experts in the field who have investigated this subject area and developed tools or approaches. Presenters included a Calvin Kam and Tony Rinella from SBI Intl, John Messner from Penn State University and Paul Hill from ARUP. I also had the chance to present past work on model evolution assessment. The team from SBI presented a initial conceptual framework to relate the various levels of analysis, including industry, organization, project and individual and how to prompt and track progress. The overall objective suggested for the initiative was “to harmonize and standardize a global buildingSMART Metrics Framework to study the Performance of organizations and projects in integrating buildingSMART standards, compliance, adoption, use, and innovation and preparedness of service supply chain members.” John Messner presented Penn State’s seminal work in the development of the BIM Capability Maturity Model and the organizational assessment tool. Paul Hill presented ARUPs tool, derived from Penn State’s work, and its deployment across the companies many locations and on many projects. Two working sessions were held to involve participants and gain perspective on the various avenues for benchmarking and performance assessment that had been developed or used by attendees. Ultimately, the outcomes of the session were the consensus on the need to further investigate and develop this initiative within buildingSMART International. Given the fact that this was a high priority item identified within the Canadian Roadmap to BIM, it was agreed that buidlingSMART Canada would remain involved and help this initiative along in any way possible.

Next steps for this initiative include: developing bSI working groups to further discuss and identify performance indicators and to start harmonizing best practice of other evaluation systems. The intent is to have a first version of sample global buildingSMART Metrics Framework for the Tokyo bSI Summit in October 2018. For people interested in participating in this initiative, contact [email protected].

Comments are closed.