Sara Beery got here to MIT as an assistant professor in MIT’s Department of Electrical Engineering and Computer Science (EECS) desperate to concentrate on ecological challenges. She has fashioned her research profession around the chance to use her expertise in computer vision, machine learning, and data science to tackle real-world issues in conservation and sustainability. Beery was drawn to the Institute’s commitment to “computing for the planet,” and got down to bring her methods to global-scale environmental and biodiversity monitoring.
Within the Pacific Northwest, salmon have a disproportionate impact on the health of their ecosystems, and their complex reproductive needs have attracted Beery’s attention. Annually, tens of millions of salmon embark on a migration to spawn. Their journey begins in freshwater stream beds where the eggs hatch. Young salmon fry (newly hatched salmon) make their strategy to the ocean, where they spend several years maturing to maturity. As adults, the salmon return to the streams where they were born so as to spawn, ensuring the continuation of their species by depositing their eggs within the gravel of the stream beds. Each female and male salmon die shortly after supplying the river habitat with the following generation of salmon.
Throughout their migration, salmon support a big selection of organisms within the ecosystems they go through. For instance, salmon bring nutrients like carbon and nitrogen from the ocean upriver, enhancing their availability to those ecosystems. As well as, salmon are key to many predator-prey relationships: They function a food source for various predators, resembling bears, wolves, and birds, while helping to regulate other populations, like insects, through predation. After they die from spawning, the decomposing salmon carcasses also replenish helpful nutrients to the encompassing ecosystem. The migration of salmon not only sustains their very own species but plays a critical role in the general health of the rivers and oceans they inhabit.
At the identical time, salmon populations play a very important role each economically and culturally within the region. Industrial and recreational salmon fisheries contribute significantly to the local economy. And for a lot of Indigenous peoples within the Pacific northwest, salmon hold notable cultural value, as they’ve been central to their diets, traditions, and ceremonies.
Monitoring salmon migration
Increased human activity, including overfishing and hydropower development, along with habitat loss and climate change, have had a major impact on salmon populations within the region. Consequently, effective monitoring and management of salmon fisheries is essential to make sure balance amongst competing ecological, cultural, and human interests. Accurately counting salmon during their seasonal migration to their natal river to spawn is crucial so as to track threatened populations, assess the success of recovery strategies, guide fishing season regulations, and support the management of each business and recreational fisheries. Precise population data help decision-makers employ the very best strategies to safeguard the health of the ecosystem while accommodating human needs. Monitoring salmon migration is a labor-intensive and inefficient undertaking.
Beery is currently leading a research project that goals to streamline salmon monitoring using cutting-edge computer vision methods. This project suits inside Beery’s broader research interest, which focuses on the interdisciplinary space between artificial intelligence, the natural world, and sustainability. Its relevance to fisheries management made it an excellent fit for funding from MIT’s Abdul Latif Jameel Water and Food Systems Lab (J-WAFS). Beery’s 2023 J-WAFS seed grant was the primary research funding she was awarded since joining the MIT faculty.
Historically, monitoring efforts relied on humans to manually count salmon from riverbanks using eyesight. Previously few a long time, underwater sonar systems have been implemented to help in counting the salmon. These sonar systems are essentially underwater video cameras, but they differ in that they use acoustics as an alternative of sunshine sensors to capture the presence of a fish. Use of this method requires people to establish a tent alongside the river to count salmon based on the output of a sonar camera that is connected to a laptop. While this technique is an improvement to the unique approach to monitoring salmon by eyesight, it still relies significantly on human effort and is an arduous and time-consuming process.
Automating salmon monitoring is crucial for higher management of salmon fisheries. “We want these technological tools,” says Beery. “We will’t sustain with the demand of monitoring and understanding and studying these really complex ecosystems that we work in without some type of automation.”
As a way to automate counting of migrating salmon populations within the Pacific Northwest, the project team, including Justin Kay, a PhD student in EECS, has been collecting data in the shape of videos from sonar cameras at different rivers. The team annotates a subset of the information to coach the pc vision system to autonomously detect and count the fish as they migrate. Kay describes the technique of how the model counts each migrating fish: “The pc vision algorithm is designed to locate a fish within the frame, draw a box around it, after which track it over time. If a fish is detected on one side of the screen and leaves on the opposite side of the screen, then we count it as moving upstream.” On rivers where the team has created training data for the system, it has produced strong results, with only 3 to five percent counting error. That is well below the goal that the team and partnering stakeholders set of not more than a ten percent counting error.
Testing and deployment: Balancing human effort and use of automation
The researchers’ technology is being deployed to watch the migration of salmon on the newly restored Klamath River. 4 dams on the river were recently demolished, making it the biggest dam removal project in U.S. history. The dams got here down after a greater than 20-year-long campaign to remove them, which was led by Klamath tribes, in collaboration with scientists, environmental organizations, and business fishermen. After the removal of the dams, 240 miles of the river now flow freely and nearly 800 square miles of habitat are accessible to salmon. Beery notes the just about immediate regeneration of salmon populations within the Klamath River: “I believe it was inside eight days of the dam coming down, they began seeing salmon actually migrate upriver beyond the dam.” In a collaboration with California Trout, the team is currently processing recent data to adapt and create a customized model that may then be deployed to assist count the newly migrating salmon.
One challenge with the system revolves around training the model to accurately count the fish in unfamiliar environments with variations resembling riverbed features, water clarity, and lighting conditions. These aspects can significantly alter how the fish appear on the output of a sonar camera and confuse the pc model. When deployed in recent rivers where no data have been collected before, just like the Klamath, the performance of the system degrades and the margin of error increases substantially to 15-20 percent.
The researchers constructed an automatic adaptation algorithm throughout the system to beat this challenge and create a scalable system that will be deployed to any site without human intervention. This self-initializing technology works to mechanically calibrate to the brand new conditions and environment to accurately count the migrating fish. In testing, the automated adaptation algorithm was capable of reduce the counting error right down to the ten to fifteen percent range. The advance in counting error with the self-initializing function signifies that the technology is closer to being deployable to recent locations without much additional human effort.
Enabling real-time management with the “Fishbox”
One other challenge faced by the research team was the event of an efficient data infrastructure. As a way to run the pc vision system, the video produced by sonar cameras should be delivered via the cloud or by manually mailing hard drives from a river site to the lab. These methods have notable drawbacks: a cloud-based approach is restricted resulting from lack of web connectivity in distant river site locations, and shipping the information introduces problems of delay.
As a substitute of counting on these methods, the team has implemented a power-efficient computer, coined the “Fishbox,” that will be utilized in the sphere to perform the processing. The Fishbox consists of a small, lightweight computer with optimized software that fishery managers can plug into their existing laptops and sonar cameras. The system is then able to running salmon counting models directly on the sonar sites without the necessity for web connectivity. This permits managers to make hour-by-hour decisions, supporting more responsive, real-time management of salmon populations.
Community development
The team can also be working to bring a community together around monitoring for salmon fisheries management within the Pacific Northwest. “It’s just pretty exciting to have stakeholders who’re smitten by having access to [our technology] as we get it to work and having a tighter integration and collaboration with them,” says Beery. “I believe particularly if you’re working on food and water systems, you wish direct collaboration to assist facilitate impact, since you’re ensuring that what you develop is definitely serving the needs of the people and organizations that you just are helping to support.”
This past June, Beery’s lab organized a workshop in Seattle that convened nongovernmental organizations, tribes, and state and federal departments of fish and wildlife to debate the usage of automated sonar systems to watch and manage salmon populations. Kay notes that the workshop was an “awesome opportunity to have everybody sharing other ways that they are using sonar and serious about how the automated methods that we’re constructing could fit into that workflow.” The discussion continues now via a shared Slack channel created by the team, with over 50 participants. Convening this group is a major achievement, as lots of these organizations wouldn’t otherwise have had a possibility to come back together and collaborate.
Looking forward
Because the team continues to tune the pc vision system, refine their technology, and have interaction with diverse stakeholders — from Indigenous communities to fishery managers — the project is poised to make significant improvements to the efficiency and accuracy of salmon monitoring and management within the region. And as Beery advances the work of her MIT group, the J-WAFS seed grant helps to maintain challenges resembling fisheries management in her sights.
“The incontrovertible fact that the J-WAFS seed grant existed here at MIT enabled us to proceed to work on this project once we moved here,” comments Beery, adding “it also expanded the scope of the project and allowed us to keep up energetic collaboration on what I believe is a very essential and impactful project.”
As J-WAFS marks its tenth anniversary this 12 months, this system goals to proceed supporting and inspiring MIT faculty to pursue revolutionary projects that aim to advance knowledge and create practical solutions with real-world impacts on global water and food system challenges.
