Research and Development Office

Improving Fish Species Recognition and Tracking Algorithms to Identify Fishes in Turbid Water

Project ID: 3618
Principal Investigator: Donald Portz
Research Topic: Ecosystem Needs
Funded Fiscal Years: 2011
Keywords: None

Research Question

Trawls and other types of netting presently form the foundation of fish monitoring in many rivers, lakes, and reservoirs. Although these gear types have proved exceptionally useful, they are much less effective with patchy or rare species. Moreover, the recent decline in some fish populations has led to concern over lethal "take" (Endangered Species Act [ESA]) by trawling methods. Recent progress in towed video imaging systems may provide a supplemental method that could be used to examine pelagic fish distribution and abundance. The potential use of underwater video cameras retrofitted to trawl surveys as a new nonlethal tool to measure the abundance and distribution of pelagic fishes has gained recent interest. We have developed an intelligent system capable of counting fish-sized objects, and work is being done currently to improve upon that system with an improved algorithm that can determine types of pelagic fishes in real time. The focus is to improve upon the existing algorithm, answering the question:

Will improvements of the fundamental subsystems in the vision algorithm assist in development of a device that is capable of reducing lethal "take" from vulnerable populations?

Need and Benefit

With the ever increasing level of computing and visual technologies, there exists an opportunity to improve the algorithms we have developed for computationally counting and classifying. There are two portions of our system that would benefit the most from additional work: 1. the subsystem that deals with detecting moving objects, and 2. the orientation or pose estimator within the classification system that determines the class or species of fish in question.

Specifically, the object detection code would benefit from a more accurate way of detecting occlusions, or when one object passes directly in front of another. Although it is an easy task for human vision, computationally it requires the algorithm to know where all of the image information has been and where it is going. One way to accomplish this is through the Kanade-Lucas-Tomasi (KLT) tracking algorithm. While the current algorithm segments foreground and background and searches the image for static blobs in two dimensions, the KLT algorithm looks for "interesting" features and tracks them, computing what is called optical flow, effectively adding a third time dimension and adding robustness to the tracking algorithm. This is important because if objects are missed at this stage, they will not be accounted for at all.

Secondly, the estimation of pose is another area in which the computer must analyze all views of the fish that it captured and determine which is the best to use in the classification phase. The current system relies on the view of the fish's eye and makes assumptions about the current orientation of the fish based on the size and shape of the presumed round eye. An improved pose estimator would allow the algorithm a better opportunity for success by improving the number of images that the classifier can use.

In addition to these software improvements, the algorithm must gain experience, or training, before it becomes useful as a classification tool. This requires many samples of real fish in a real environment. Additional time in the Sacramento-San Joaquin River Delta would allow for additional training samples needed to make our system robust and reliable.

Contributing Partners

Contact the Principal Investigator for information about partners.

Research Products

Bureau of Reclamation Review

The following documents were reviewed by experts in fields relating to this project's study and findings. The results were determined to be achieved using valid means.

SmeltCam: Underwater Video Technology for Identifying and Measuring Abundance of Pelagic Fishes (final, PDF, 2.1MB)
By Donald Portz
Report completed on April 30, 2012

Return to Research Projects