• News

A brave new world … combining VR with human operators and Machine Learning for improved forest inventory practices

During the past few years, FWPA has supported researchers testing new technologies to improve various elements of the forestry and wood products supply chain.

Some of the more time-consuming and labour-intensive tasks in forests relate to field inventory practices. The likes of manual tree stem assessments not only demand a lot of person-power, but also often need to be conducted in difficult to access areas and in the presence of various hazards such as falling branches, or conditions on the ground that might lead to trips, slips or falls.

Developments from new research, which combines Virtual Reality (VR) operated by humans and using Machine Learning (ML) algorithms, could represent a major step forward in improving forest inventory practices.

The results of this work have recently been published and represent the latest in a series of research projects designed to look at how technology might revolutionise the way field inventory practices are managed. Benefits including improved worker safety, better access to hard-to-reach areas and reduced labour costs.

ML is essentially a form of Artificial Intelligence (AI) and computer science that uses data and algorithms to imitate how human beings learn, gradually improving the accuracy of whatever ‘machine’ is doing the ‘learning’.

This latest project has built on progress made during initial VR work as part of a previous FWPA-funded pilot project, conducted by a team at the University of Tasmania (UTAS).

The researchers involved in that project used dense point cloud data captured from a pre-harvest Pinus radiata inventory plot by helicopters and unmanned aerial vehicles (UAVs). This data was then fed into an immersive VR environment using commercially available hardware, making it possible to investigate the potential of VR to replace current forest inventory fieldwork approaches.

Lead Researcher Dr Winyu Chinthammit of the Human Interface Technology Laboratory at the University of Tasmania said the project was prompted by forests becoming more difficult to evaluate manually due to increasing safety concerns and access problems.

“We were looking to work with people who didn’t have any experience of VR, but who would traditionally go into forests to assess trees, in order to test how people who already have skills in forestry might perform inside a VR environment,” Chinthammit said.

Experiments and testing with industry volunteers found all participants demonstrated the capacity to work within the VR environment, and most were able to successfully use the tools and take basic tree measurements, including diameter and height, within a reasonable time.

However, more complex measurements relating to features such as wobble, sweep type, stem damage and branch size inside the canopy were not as accurate.

“This project provided evidence that a field crew can indeed operate inside a VR environment and perform some of the same basic tasks they would normally complete as part of their everyday operations on the ground,” Chinthammit said.

“Going forward, we plan to use more advanced visualisation and rendering, in combination with ML techniques, to determine how the data can be used to give a more detailed structural view of the canopy.

“The ultimate aim is for users to assess forests in a VR environment with the same levels of accuracy that would be achieved had they gone into the forest themselves.”

This technology has been further developed by the team using the results of extensive user testing and feedback undertaken in collaboration with several industry partners, including field operators themselves.

Additionally, researchers involved with the most recent project took advantage of the development of automated point cloud processing algorithms for individual tree detection and segmentation, developed through FWPA-funded research led by Dr Mitch Bryson and his team at the University of Sydney. The researchers also utilised the outcomes of work conducted and funded by the National Institute for Forest Products Innovation (NIFPI).

The latest project investigated the concept of the Human-in-the-loop (HITL) framework. Essentially, HITL occurs where humans work in collaboration with machines to perform tasks, combining their abilities to ensure the best results.

In this context, HITL saw human VR users in the forest directly collaborate with the automated Machine Learning (ML) point cloud processing algorithms. So, for the time being, HITL provides a framework for VR users to collaborate with the ML engine when making tree assessments. This allows human field operators to interchange knowledge of stem segmentation with machine-learning algorithms.

Using this approach, the ML algorithms developed during this project demonstrated continuous advancement. Combining VR human operators with ML algorithms in this way has demonstrated the potential to significantly enhance the capabilities of the ML engine when it comes to estimating accurate forest inventory measurements, safely and in hard-to-access areas.

However, the researchers say there remains challenges for the HITL concept to become fully practical.

Posted Date: November 1, 2023

Related Resources

A-leap-forward-reducing-the-energy-and-climate-impacts-of-creating-paper-products
  • News

New research has shown that genetically modified trees with less lign…

NASA-GEDI-space-lasers-and-.-forestry
  • News

An international research team using data obtained by NASA – ye…