Discover millions of ebooks, audiobooks, and so much more with a free trial

Only $11.99/month after trial. Cancel anytime.

Artificial Intelligence in Earth Science: Best Practices and Fundamental Challenges
Artificial Intelligence in Earth Science: Best Practices and Fundamental Challenges
Artificial Intelligence in Earth Science: Best Practices and Fundamental Challenges
Ebook918 pages7 hours

Artificial Intelligence in Earth Science: Best Practices and Fundamental Challenges

Rating: 0 out of 5 stars

()

Read preview

About this ebook

Artificial Intelligence in Earth Science: Best Practices and Fundamental Challenges provides a comprehensive, step-by-step guide to AI workflows for solving problems in Earth Science. The book focuses on the most challenging problems in applying AI in Earth system sciences, such as training data preparation, model selection, hyperparameter tuning, model structure optimization, spatiotemporal generalization, transforming model results into products, and explaining trained models. In addition, it provides full-stack workflow tutorials to help walk readers through the whole process, regardless of previous AI experience. The book tackles the complexity of Earth system problems in AI engineering, fully guiding geoscientists who are planning to implement AI in their daily work.
  • Provides practical, step-by-step guides for Earth Scientists who are interested in implementing AI techniques in their work
  • Features case studies to show real-world examples of techniques described in the book
  • Includes additional elements to help readers who are new to AI, including end-of-chapter, key concept bulleted lists that concisely cover key concepts in the chapter
LanguageEnglish
Release dateApr 27, 2023
ISBN9780323972161
Artificial Intelligence in Earth Science: Best Practices and Fundamental Challenges

Related to Artificial Intelligence in Earth Science

Related ebooks

Earth Sciences For You

View More

Related articles

Reviews for Artificial Intelligence in Earth Science

Rating: 0 out of 5 stars
0 ratings

0 ratings0 reviews

What did you think?

Tap to rate

Review must be at least 10 words

    Book preview

    Artificial Intelligence in Earth Science - Ziheng Sun

    Chapter 1: Introduction of artificial intelligence in Earth sciences

    Ziheng Suna,b; Nicoleta Cristeac,d    a Department of Geography and Geoinformation Science, George Mason University, Fairfax, VA, United States

    b Center for Spatial Information Science and Systems, George Mason University, Fairfax, VA, United States

    c Department of Civil and Environmental Engineering, University of Washington, Seattle, WA, United States

    d eScience Institute, University of Washington, Seattle, WA, United States

    Abstract

    There are many compelling visions about how AI should look like in the future of Earth sciences. By examining the past, we might identify potential future trends. This chapter will revisit the dramatic development of AI techniques, overview the endeavors Earth scientists and computer scientists have gone through, summarize the successes and failures, and shed light on the path on possible ways forward during upcoming decades. It also explains the motivation of writing this book and how it can assist Earth scientists to understand, accept, and use AI in conducting the impossible research in the past.

    Keywords

    Earth system sciences; Artificial intelligence; Machine learning; Nature hazards

    1: Background and motivation

    When it comes to the concept of the Earth system, the first picture on most people's minds might be earthquakes, because they are the common and direct consequences of Earth system movements. About earthquakes, Dr. Sun has his own story to tell:

    When I was a kid (in the 1990s), an earthquake hit my hometown at the northeast edge of the Tibet plateau. I remember vividly the ceiling light hanging from a soft line was swinging wildly, and the earth jolted, shivered, and tried to get away from my feet. I attempted to stand up but the shake transmitted to my legs and dysfunctional them completely. I found myself crawling on the ground and couldn’t move a finger. My hands grasped the earth so tight that I could feel every stroke of the quake waves. The quake fades away after several minutes. As a middle schooler, I had no idea how close death can be and how things would go from there. I was fortunate as the ceiling stopped coming down and the brick walls still held up. It is one of the most memorable life-or-death experiences and the feeling of that kind of helplessness is still vivid after twenty years. After I left home for college, several other major earthquakes hit that region repeatedly, including some devastating ones: Wenchuan quake (magnitude-8.0, 2008, der Hilst, 2008) and Yushu quake (magnitude-7.1, 2010, Chen et al., 2010) (as shown in Fig. 1). Tens of thousands of people like you and me died.

    Every day natural hazards like earthquakes test the resilience of humanity and cause the same kind of helplessness in many entities from individuals to the metropolitan (Hyndman and Hyndman, 2016). After surviving millions of incidents and having evolved with accumulated knowledge over thousands of years, humankind has become the dominant species on this planet. The power of the entirety of civilized society makes its own path to impact the Earth's environment. We have built skyscrapers, towers, homes, roads, bridges, factories, ships, planes, railways, satellites, power grids, scales, to improve our lives and connect billions of people anywhere anytime. However, all the infrastructure and the supplies supporting our civilization totally depend on the stability and smooth cycles of the Earth systems. Any disturbance or disruption of the Earth cycles could have devastating effects. Recent trends in the climate and the increase of radical events have caused tremendous amounts of damage and raised concerns among Earth scientists (Houghton et al., 1990). Although the threats from nature have been greatly reduced after robust solid buildings were built to endure large earthquakes and weather forecasting systems were developed to prevent casualties from hurricanes, snow, floods, landslides, and tsunami, we need to remain vigilant as the dangers still persist and even increase in intensity and frequency.

    Individual natural disasters may be deadly but their impacts are relatively small scale or affect one or two regions which could recover. If we lay eyes on the long-term sustainability of the whole human society in the next hundred or even a thousand years, there are three grand challenges that are global and likely nonrecoverable: food security, climate change, and energy resources (Gregory et al., 2005). It is hard for most people today to believe but it is a fact that hunger is still happening around the world. Food shortage or insecurity has close relation to poverty but in a bigger picture, is the direct result of the changes in Earth systems. Enduring extreme drought, deforestation, emissions and chemical soil pollutants, are all contributing together, directly or indirectly impacting our food supply chain from upstream agriculture to food industries. As the world population foreseeably surpasses ten billion during the next decade—and some forecasts predicting twenty billion by 2050—the strain on our food systems will become a critical problem as today's agriculture can’t deliver enough food to meet that need. Disruptive climate events have been more frequent in recent years, likely caused by the increasing emissions funneled by the globalization of industries and commercial activities. Worsening air quality could be even more harmful than food shortage. Many forms of atmospheric pollution are constantly emitted and affect public health and the environment. Global circulation patterns (Oort, 1983) can transport the pollution rapidly around the world. Humanity has fought numerous wars to acquire natural resources, from land to water, from minerals to oil, from forest to grass. We continuously exploited our planet for resources to support our development. However, the natural resources that can produce energy are limited and will eventually be used up due to the speed of consumption being much faster than its creation. Besides meeting the food problem of the exploding population, meeting energy needs is also a long-term challenge that we or future generations will have to face sooner or later. The ongoing energy crisis and rocketing high oil and gas prices could cause global conflicts over energy in the next few years.

    Science and technology play significant roles in tackling both the individual natural hazards and the long-term survival challenges of humankind. For instance, learning about atmospheric pollution would be valuable knowledge to prevent further deterioration of our air quality. Scientists made great breakthroughs on understanding the mechanisms of natural hazards, and built computer models to simulate and predict future events and their impacts. Based on aquired knowledge, new technologies have been invented and used to mitigate damages and sustain our society: felling of trees to control wildfire spread, use of antiquake designs to increase home resilience to earthquakes, and application of smart irrigation schedules to maximize crop yields while saving water, among others. In the past decades, science and technology have become the backbone of most solutions to our most urgent crises and will continue to be our most valuable tools in the next chapter of human development.

    However, the development of science and technology is not straightforward and is always accompanied by many myths, assumptions, theories, and technical limitations throughout its development. The journey of scientific research is not always successful and has gone through ups and downs. Take climate prediction models as an example. There are so many chemical reactions to consider and not all of them are revealed and studied in the existing research. Scientists have conducted countless experiments and improvements to push the models to be more accurate and closer to reality and have achieved incredible progress. However, due to the model complexity, the computing of each model run takes huge amounts of resources and in turn could negatively affect the environment resulting from the energy consumption. In addition, even with high performance computing resources, the reliability of the model results are still in doubt, especially those long-term prediction results. The short-term prediction, or forecasts within 3 months, can be constantly corrected and improved using the near-real-time collected observations from large-scale sensor networks. For long-term forecasts, especially for those catastrophic events in the coming years, the capability of the current models still falls short of our expectation. Scientists are actively seeking for new environment-friendly solutions to advance model simulation and prediction of atmospheres and all the other spheres in the Earth system (Alley et al., 2019). Artificial Intelligence (AI) technologies have been aggressively experimented with in Earth system sciences and attempted to solve these urgent problems and provide a solution for those ultimate challenges of humanity.

    2: AI evolution in Earth sciences

    AI is not a new technology, and is historically closely connected to other domains such as mathematics, astronomy, and physics. The basic concepts and algorithms were invented a long time ago. Neural networks were first invented in 1944 by Warren McCullough and Walter Pitts (Hardesty, 2017). The optimization backbone algorithm backpropagation was first developed in the 1970s (the basics were formed in the 1960s or earlier and there is an argument about who is its real inventor—Synced, 2020). Stochastic gradient descent used in the back propagation was first described by Robbins and Monro in A Stochastic Approximation Method (Robbins and Monro, 1951), and later Kiefer and Wolfowitz introduced the machine learning variant, Stochastic Gradient Descent, in their paper Stochastic Estimation of the Maximum of a Regression Function (Kiefer and Wolfowitz, 1952). Convolutional neural networks were first introduced in the 1980s by Yann LeCun when he was a postdoctoral researcher. The recurrent neural network was designed in 1986. Long short term memory, one of the most successful industrial workhorses for sequence data processing like speech recognition and translation, was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber. However, in 1980–1990s, most of the neural network models were limited by the computing power and memory volume back then, and it took a very long time to train the model and the cost was too high and almost intolerable for individual scientists. AI research in Earth sciences also faces the same challenge and it went through another AI winter. After entering the 2010s, with the rapid development of computer hardware and according to Moore's Law (the number of transistors on a dense integrated circuit will double in about every two years), training AI models with more layers has become realistic. The famous ImageNet competition (Deng et al., 2009), an annual global AI competition event hosted by researchers from Stanford, has shown us that the scale of the model is one of the keys to unlock the real power of AI to pursue advanced intelligence. Deep learning has become one of the most popular research areas and could almost be seen in every branch of Earth sciences today. The idea behind deep learning is based on adding more hidden layers to the neural network, using a series of methods to control the model regularization and balance between overfitting and underfitting. In every iteration, the difference is calculated and the back propagated (or back optimized by another algorithm) to adjust the weights on the connections between the neurons which could be located on two sequential layers or two nonsequential layers (ResNet, U-Net). Numerous networks deal with specific types of datasets. Convolutional neural networks are one of the most widely used models to learn patterns from images or n-dimensional gridded data, a common task in Earth system sciences. Most powerful neural networks of today are larger than those of even just a few years ago, such as GPT-3 (Generative Pre-trained Transformer 3) (Brown et al., 2020) which has 96 layers with about 175 billion trainable parameters. Training the models takes a huge amount of computing hours and specialized hardware like GPU or TPU for greater benefits of parallelism to reduce overall time costs. Training big models on Earth scientific datasets could easily go up to hundreds of Terabytes or even Petabyte level could cause challenging problems such as running out of memory and or excessively long computation times.

    The development of AI in Earth sciences is still at an early stage. Despite some challenges, researchers have managed to peek at the potential of AI when applied to solve complex problems such as weather and climate prediction, or land cover land use mapping, wildfire detection, ocean eddy detection, earthquake signal capture, hurricane trajectory forecasting, and others (Sun et al., 2022). AI has shown its excellence in shedding light on use cases where human data analysts have trouble interpreting the information. At present scientists rely on numerical physics-based models which integrate sophisticated mathematical equations to calculate the desired results given certain conditions and assumptions. For example, given the temperature, humidity, precipitation, wind direction, of the whole region, the model will use them as initial conditions and simulate the situation's evolution to deliver a prediction. It is more intuitive to tune the model parameters as we have some knowledge about their variations and impacts on internal mechanisms. The AI model provides a solution which appears to be simpler than investing too much time in tuning the traditional model to fit Earth observation and data collection. Building AI models doesn’t require the modelers to have superb mathematical background and deep understanding of those equations. Once the hardware and software environment is set up, the models will learn patterns from the data perspective. The drawbacks are also obvious: AI can be easily influenced by the quality of the input data, and more vulnerable to noise in the training data. Interpreting the results could become difficult, which is one of the main reasons the credibility and reliability of AI models are constantly questioned.

    There are few use cases of AI being operationally applied in Earth data product generation and application at present. Most existing use cases focus on land cover mapping. For example, Esri Inc. contracted Impact Observatory to generate global annual land cover maps from 2017 to 2021 at 10 m resolution using AI to train on ESA (European Space Agency) Sentinel-2’s six spectral bands. They used billions of human labeled image pixels to complete the training of an AI land classification model of 9 classes including vegetation types, bare surface, water, cropland and built areas. The efforts behind the scene are massive and take a lot of human hours to create those labels for training. The cost is highly expensive depending on the scope of the problem and target application extent. For example, creating a global agriculture map at 10 m resolution will probably take ten times or even more effort than the global land cover map with nine classes, because the crops have way more classes and need more expertise for the human labellers to distinguish the crops from satellite images. Some crop types are not recognizable from satellite images and need ground survey instead which will drive the overall cost even higher. The same problem is especially true in other domains in geosciences as well, such as geological mapping which requires the labelers to annotate accurate labels on rocks and soil to match with the continuous unmanned observations such as the satellite or drone survey images to create reliable geology maps. Many researchers are looking for alternative solutions such as using crowdsourcing or citizen scientists to help create those labels. However, the existing efforts and the datasets resulting from those activities only covers a tiny portion of all the training labels needed to address more critical problems in Earth sciences. The bottleneck is the availability of training label datasets that can be used to correspond with the huge datasets provided by the continuous observing sensors in places. Even when labeled datasets are available, there is still a huge amount of work to transform them into AI-ready data format, such as the decades of work on plants and animals whose records are written in thousands of books and reports. Digitalizing them and turning them into structured, unified, and analysis-ready form would also be very challenging.

    3: Latest developments and challenges

    AI has been attracting attention for years and exciting breakthroughs happened across multiple Earth science domains. Geoscientists have made a lot of attempts to bring in AI technologies to address challenging problems in the past few decades. Much progress has been achieved across domains, while a series of questions and challenges still remain.

    In seismology, AI helped to pick up early earthquake and volcano seismic signals from the giant set of wave signals and tell the differences between valid signals and noises. The use of AI can be found in all kinds of research cases: earthquake detection and phase picking, earthquake early warning, ground motion prediction, seismic tomography, and earthquake geodesy. LANL held a high-profile competition on Kaggle for earthquake prediction and attracted many machine learning practitioners (Johnson et al., 2021). Many ML algorithms, such as LSTM, XGBoost, Random Forest, and genetic programming, have been widely experimented on many datasets. AI-enhanced approaches can detect earthquake-related signals and early signs from the noises and distinguish them by accurately and objectively determining the noise thresholds. The traditional way requires experts to visually filter and identify those earthquake signals using phase association methods, which undoubtedly delays the data processing and slows down the information extraction phases. These traditional methods have challenges in detecting small seismic events that might be important and decisive in predicting the future big events. AI methods are pursued mainly because (1) they are expected to extract more patterns and relationships between collected datasets and accumulated prior knowledge; (2) they can significantly improve computational efficiency of seismic model processing. However, there are two major challenges in processing seismic data to detect or predict hazard events such as earthquakes and volcanoes (Jiao and Alavi, 2020): (1) there are massive amounts of noises in seismic data; (2) there are many undetected events (e.g., small earthquakes, or earthquakes in remote areas with less sensor coverage), which are likely to be excluded from training and result in biases and underfitting of AI models.

    In land cover land use mapping, AI has been extensively used to generate higher spatial–temporal resolution maps (Sun et al., 2019). AI can learn from the collected tremendous amount of human labels in the past decades and use the hidden patterns, some of which are yet to be discovered by scientists, to classify the newly captured observations (satellite images), or to map the land cover in a near-real-time manner. For instance, the living atlas project (Wright et al., 2022) generates annual 9-class global land cover maps using billions of human-labeled samples. There are several main driving factors behind the AI efficiency in this domain that uniquely stands out and is more generally welcomed and accepted by the community. Earth observing satellites have captured petabytes of datasets in the past five decades and will continue to increase at an unprecedented rate after commercial companies join in the market. Most imagery datasets collected by government agencies are openly available. The science outreach teams within NASA, NOAA, USGS, and ESA have devoted significant effort to make those datasets easily accessible and usable via a lot of websites, software tools, standardized web services, and programming libraries. Along with the increase in the number of satellites, the spatial–temporal-spectral resolution of the images has dramatically improved. The large objects on the Earth surface such as forests, lakes, mountains, rivers, icebergs, etc., can be clearly seen and analyzed visually by experts. The availability and high resolution of satellite images makes it very easy to create labels about general land cover classes without having a field survey. That is why creating billions of human labeled samples is practical. More importantly, many research groups made their training labels open to avoid huge amounts of duplicated labeling efforts by the community, allowing researchers to worry less about time-consuming labeling work and focus more on AI model improvement. Also, satellite image classification is very similar to the classic AI/ML tasks, which were originally designed for image classification, like images of dogs and cats found in ImageNet competitions. It is now easy to adapt the experiment setup, with expanded classification goals. Relatively speaking, due to the availability of satellite images and training labels, most AI experiments in land cover classification can be redone by a new group in a short period. In other words, such research has a lower barrier of entry for beginners of either AI or geosciences.

    In hydrology, AI is starting to takle problems related to ambiguity and uncertainty in hydrologic predictions or scaling problems of regional models. Hydrology-related scientific challenges are further complicated by the noisy, complex, and dynamic nature of variables such as groundwater, evapotranspiration, discharge, sediment transport, and their interaction with soil and climate. Time series prediction models have been widely applied to address hydrology specific questions, e.g., long short term memory (LSTM) approaches. The models can find patterns in the existing datasets that scientists have collected for decades. Nonlinear models may perform better than linear models. ML has been successfully used in flood forecasting, precipitation, water quality, and groundwater estimation (Sun et al., 2022).

    However, there are a number of pitfalls with the current AI techniques. Not all the Earth scientific challenges that are in need for answers have rich data suitable to train an unbiased AI model, and the current AI techniques are not well equipped to deal with data-scarce scenarios. For problems with limited datasets, AI cannot infer the patterns but instead may learn the patterns and noises in the restricted samples, which may not align with the existing knowledge. At the same time, the massive amount of human-labeled samples required are another bottleneck problem preventing AI from reaching the practical level of application. For tasks like simple hierarchy and land cover mapping, it is possible to quickly create lots of labels just based on the satellite images. However, for complex classification tasks such as agricultural crop mapping or detailed vegetation classification, it may not be possible to create labels simply based on remote sensing data to avoid doing field surveys. AI needs the input and output data to refer to the same time period to create a good match. Using data from various sources may pose difficulty in creating valid matches for AI. For instance, the history of satellite observations can be traced back to the 1970s and the continuity of data coverage and quality varies. The ground surveys could be conducted during periods which might fall out of the observation window of the satellite images. For example, the ground occurs on April 10, but cloudless satellite images are only available on Mar 20 and May 1. Using the data on either dates will result in the models that may represent inadequate patterns, missing the growing stage. High quality training data pairs are critically important but not guaranteed by data providers due to the limitation of nature (weather, clouds) and observation capabilities (revisiting period, spatial resolution, swath width, etc.).

    4: Short-term and long-term expectations for AI

    AI is not a magic tool. It has many limitations and as a result experiences many restrictions and other challenges during operational use. Classic AI includes tasks such as image object recognition, car plate recognition, human face matching, tabular data prediction, but we still have to treat AI as a tool instead of an intelligent entity. Most AI systems can be considered as an improved version of the current rule-based expert systems but still have a long way to reach complete self-learning intelligence, or artificial general intelligence (AGI) (Goertzel, 2007). This section will describe the general goals (or our expectations) of AI research progress in Earth system sciences from both short-term and long-term perspectives.

    The general short-term goals for AI practitioners in the next two decades will likely be: (1) improve models' accuracy to the expected level; (2) stabilize models' performance over space and time; (3) operationalize AI models in real-world practice (e.g., natural hazard responding). The first goal is obviously that AI first needs to be accurate enough before it can be considered usable. Current AI models are still struggling to reproduce the expected accuracy when confronted with new data, especially data representing patterns or distributions that are not directly included in the original training datasets. A common challenge for AI models is to address the overfitting and underfitting. The current proposed approaches such as cross validation (e.g., GridSearch and RandomizedSearch in scikit-learn) usually require multiple attempts to find the best hyperparameter configuration and are not very efficient. AutoML automates the model selection, parameter tuning, and model comparison, which significantly reduces the requirements for manual tuning.

    The long-term expectations of Earth AI in this century are quite exciting to imagine and vary by domain. The ideal breakthrough will likely happen on perfecting model reliability and consistency. A model could stably produce accurate predictions without any human intervention, or a model can identify the noise in the inputted signals and self-adjust to adapt to the abrupt changes in the real world. People will no longer have to write multiple scenario-dependent rules to restrict model behaviors. Models' spatial–temporal capability will be greatly enhanced to enable humans to enter a previously untouched territory. Take hurricane prediction as an example. The current prediction of the trajectory may evolve to a new service with very high accuracy about its path, wind speed, and precipitation in each county, or even at neighborhood level. Homeowners could receive warning notices days ahead to take actions to protect their properties. First responders could quickly identify the most damaged areas and allocate resources accordingly in the most efficient manner. Some daring ideas could envision how to turn a disaster into a potentially constructive event, e.g., developing new airborne (floating) electricity generators and placing them on the path of hurricanes to collect the tremendous power of nature. We could imagine that in the next century, human beings will very likely turn the situation from passively reacting to natural disasters into actively predicting and eventually taking advantage of the extreme natural events. AI will definitely be one of the most fundamental tools to bring that beautiful vision into reality.

    5: Future developments and how to adapt

    The recent developments in AI are rapid and dramatic and the pace of progress is still expected to accelerate further in the next decade. There are many development directions in the AI world, and all the users from Earth sciences need to prepare to adjust their strategy and adapt to the upcoming changes. Research groups or AI companies will upgrade their AI models to more powerful and cutting edge models. Models can be easily switched as most AI modules in operational systems are self-contained and treated as a black box. One of the challenging tasks during this transition is to evaluate the new changes in stability, reliability, accuracy, noise resilience, and time costs. We have witnessed that the benefits brought by new models are often overshadowed by the high cost, slow turnaround, high vulnerability to noise, and unexplainable results. To make the model change easier and smoother, the full stack AI workflow needs to stay consistent over time, with the data preparation and postprocessing steps standardized and the goals of AI models kept them unchanged. A comprehensive evaluation of the new model must be conducted to justify its replacement of the old model.

    The development of software and services will be limited without the development of the underlying hardware. Moore's Law (the number of transistors on a square inch of integrated circuit will double every year) has been validated over the past fifty years but recently the chip progress has seen a slowdown. We may find a way to continue the speedy improvements in the world of computing. The increase in computing power we experienced in the first AI boom (1980–1987) has inspired and supported the ongoing AI evolution. Foreseeably, more powerful computers, or not even digital computers (e.g., analog chips, quantum computers) will be introduced and experimented to explore new breakthroughs. It will undoubtedly bring dramatic changes to the existing AI pipelines, and some profound hardware replacement will cause the infrastructure and technology stack to be rethought and probably redesigned.

    The trends of AI are not only toward more accurate and complicated, but will likely become more portable and lightweight. Edge AI is a central concept that shifts the AI model running from data centers to personal devices like smartphones, smart cars, and robots, and makes them act as decentralized agents without communicating with external web services. In contrast to cloud AI which has everything running inside cloud data centers and only has an API exposed for users, edge AI can facilitate individual devices to run AI models. The current strategy is to deploy a pretrained model to the edge devices to consume and produce results. There are also researchers trying to use edge devices to train AI models in place, but this activity requires the AI models to be more lightweight, with fewer trainable parameters, and with a quick turnaround.

    6: Practical AI: From prototype to operation

    It is always easier said than done. There is a long way from prototype to production, and many AI researchers halt at the prototype stage and their achievements never reach production. It could be quite disappointed to learn about the real ratio of production conversion across the entire AI research community. To increase the success rate, a series of guidelines and actions must be taken throughout the entire lifecycle of AI research to prepare them for real-world adoption. The real-world application scenario is very chaotic and disorganized, which results in the violent collision of the world as imagined with the world as it is. There are many roadblocks that need to be removed to land a new model in a production environment. Here is a list of some of the major obstacles and general guidelines for solving them.

    The first common issue is data bias. Most AI models are trained on a subset of the real dataset and the pattern representation always has biases toward the majority across the various classes. Take satellite based land cover classification as an example. If a region is composed of 90% corn fields and 10% soybean fields, the model will lean toward corn because classifying a pixel as corn will have less penalty to the overall accuracy. Many solutions have been proposed to tackle the bias problem like the dropout in neural network, the class weight in Scikit-learn, intentionally augmenting the minority classes in training data, etc. There is no standard answer to this challenge and ideally it would be considered together with the application scenarios and the pattern similarities between training data and the real data to judge if the model is suitable for the use case.

    The second challenge is explicitly specifying the spatiotemporal limitation of the trained AI models. Production software has to specify its application terms and scope ahead. The patterns learnt by AI have restrictions in its applicable spatial or temporal extent as its training dataset is usually regional and for a certain period of time. Using the example of the hurricane trajectory prediction, the AI model is could be initially trained using the dataset in one region (tropical cyclones), with a fixed spatial extent. Using this model in another region and another season will likely have uncertain results, and would be discouraged. These limitations must be explicitly labeled in the delivered

    Enjoying the preview?
    Page 1 of 1