11/30/2022 McCall Macomber
A team led by Erol Tutumluer (CEE) and John M. Hart (CSL) developed an artificial intelligence approach that categorizes the size and shape of riprap within stockpiles.
Written by McCall Macomber
Grainger College engineers think outside the box by creating innovative solutions for everyday challenges.
A team led by Erol Tutumluer, Department of Civil and Environmental Engineering Abel Bliss Professor in Engineering, and John M. Hart, Coordinated Science Laboratory’s Principal Research Engineer, do just that in a joint Illinois Center for Transportation (ICT) and Illinois Department of Transportation (IDOT) project, “R27-214: 3D Image Analysis Using Deep Learning for Size and Shape Characterization of Stockpile Riprap Aggregates — Phase 2.” They are joined by Andrew Stolba, IDOT’s chief geologist.
Tutumluer and Hart developed an artificial intelligence approach that categorizes the size and shape of riprap — rock or other material used to protect shorelines against erosion — within stockpiles.
Riprap is currently inspected individually or manually weighed at quarries or construction sites before placement by shorelines. Placing improperly sized riprap by shorelines may lead to increased soil erosion or greater costs.
The two build off work they conducted in ICT-IDOT project R27-182, in which they developed a 2D approach to assess riprap.
“In 2D images you’re limited to certain views, and in a stockpile of riprap rocks you only see a partial view,” Tutumluer said. “But to do a much better job at predicting the weight of a rock, you need to go to 3D.”
The two seek to estimate the volume of individual riprap rocks in large stockpiles by determining their 3D shape. That’s where AI comes into play. Tutumluer and Hart harness computer vision — an AI field in which computers are trained to understand and interpret images or videos — to develop an image segmentation approach. Their approach will allow engineers to use smartphones to determine the 3D size and shape of individual riprap in stockpiles and eliminate the need to weigh rocks manually.
The researchers used deep learning, an AI technique, to train the system using individual riprap rocks with known volumes placed in stockpiles. They took approximately 100 images per riprap stockpile to create a point cloud — data points used to represent a 3D object — to pick out individual riprap in the stockpile and to predict their completed shapes.
To further train the algorithm, Tutumluer’s team also generated artificial riprap based on their existing samples by applying a technique used to create artificial scenery in video games.
Recent UIUC doctoral graduate Haohang Huang, who developed the AI algorithm as part of his thesis, is currently seeking a patent for their developed algorithm.
The researchers integrated the trained algorithm into a user-friendly software program for IDOT practitioners and industry members.
“We’ve created something that’s usable for an industry,” Hart said. “We had real contact with the users, and we went out in the field and saw what they experience. We were able to use that to create a practical piece of software that will really help them accomplish what they need to do out in the field.”
The next steps involve hosting training workshops on the software for industry members and practitioners led by the researchers. The project will conclude June 2023.
The researchers aim to expand their work in a future project by further developing the technology to characterize smaller aggregates used for pavement construction. They will also seek to investigate large stockpiles in quarries using drone images.
Tutumluer and Hart credit their success to Andrew Stolba and the other project personnel who assisted with the effort as well as Tutumluer’s graduate students: Haohang Huang, Jiayi Luo and Kelin Ding.
“We’re lucky to have extremely skilled and smart students at U of I,” Tutumluer said. “They helped bring computer vision and computer science technologies into this project for civil engineering solutions.”