Comparing Different Statistical Models and Pre-processing Techniques for Estimation of Soil Particles Using VNIR/SWIR Spectrum

Document Type : Research Article

Authors

• Mahboobeh Tayebi ¹
• Mahdi Naderi ¹
• jahangard mohammadi ²
• Mahdieh Hosseinjani Zadeh ³

¹ Shahrekord University

² isfahan university

³ Graduate University of Advanced Technology, Kerman

https://doi.org/10.22067/jsw.v32i1.63618

Abstract

Introduction: Soil texture is one of the majorphysical properties of soils thatplays important roles inwater holding capacity, soil fertility, environmental quality and agricultural developments. Measurement of soil texture elements in large scales is time consuming and costly due to the high volume of sampling and laboratory analysis. Therefore, assessing and using simple, quick, low-cost and advanced methods such as soil spectroscopy can be useful. The objectives of this study were to examine two statistical models of Partial Least Squares Regression (PLSR) and Principal Component Regression (PCR) to estimate soil texture elements using Visible and Near-Infrared (VNIR) and Short-Wave Infrared (SWIR) reflectance spectroscopy (400-2450nm).
Materials and Methods: A total of 120 composite soil samples (0-10 cm) were collected from the Kafemoor basin (55º 15' - 55º 25' E; 28º 51' - 29º 11' N), Sirjan, Iran. The samples were air dried and passed through a 2 mm sieve and soil texture components were determined by the hydrometer method (Miller and Keeny 1992). Reflectance spectra of all samples were measured using an ASD field-portable spectrometer in the laboratory. Soil samples were divided into two random groups (80% and 20%) for calibration and validation of models. PLSR and PCR models and different pre-processing methods i.e.First (FD) and Second Derivatives (SD), Multiplicative Scatter Correction (MSC) and Standard Normal Variate (SNV) were applied and compared to estimate texture elements. The cross‐validation method was used to evaluate calibration and validation sets in the first part (80%) and coefficient of determination (R2), Root Mean Square Error (RMSE) and Residual Prediction Deviation (RPD) were also calculated. For testing predictive models, the second part of data (20%) was used and R2 and RMSE of predictive accuracy were calculated.
Results and Discussion: The results of applying two statistical models for estimatingLogClay (%) showed that R2of calibration (R2CV) and validation (R2VAL) datasetranged from 0.22 to 0.72 and 0.12 to 0.54, respectively. The lowest RMSE was computed for PLSR model with SD pre-processing. The highest RPD of calibration (RPDCV) and validation (RPDVAL) were obtained for PLSR with SD pre-processing technique which was classified as a very good and good model, respectively. The results indicated possible prediction of soil clay content by using PCR model with SD pre-processing techniques. In addition, the PCR predicted soil texture elements poorly according to RPD values while the PLSR model with SD pre-processing was the best model for predict‌ing soil clay content. The R2CV and R2VAL of PLSR models for LogSilt (%) varied from 0.34 to 0.73 and 0.27 to 0.58, respectively. The RMSECV varied from 0.14 for FD pre-processing to 0.23 for no-preprocessing and the RMSEVAL rangedbetween 0.18 and0.24. The highest RPDCV (2.07) and RPDVAL (1.59) were obtained for PLSR with FD pre-processing which were classified as very good and good models, respectively. The results of PCR model developments for estimating LogSilt (%) indicated that the highest RPDCV and RPDVAL were, respectively, 1.31 and 1.25 for MSC pre-processing techniques which were rated as poor models. On the contrary to PLSR models, PCR models were not reliable for predicting LogSilt (%).Theresultsof PLSR models for estimatingLogSand (%) revealedthat the highest R2CV and R2VAL were 0.56 and 0.47, respectively and the lowest RMSECV and RMSEVAL were 0.14 and 0.16, respectively which were obtained for SD pre-processing. The RPDCV and RPDVAL values for SD pre-processing in PLSR model were 1.59 and 1.39 which were rated as good and poor performance of predictions, respectively. The highest RPDCV and RPDVALfor PCR models were obtained with the MSC pre-processing indicating poor model. Therefore, PLSR model with SD pre-processing techniques was superior model for estimation of LogSand(%).Overall, PLSR model with SD pre-processing techniques performed better in estimatingclay and sand and PLSR model with FD pre-processing gave better estimate of silt content.
Conclusions: Our finding indicated thatclay and silt contentcan be estimated by using electromagnetic spectrum between VNIR-SWIR region. Further, spectroscopy could be considered as a simple, fast and low cost method in predicting soil texture and PLSR model with SD and FD pre-processing seems to be more robust algorithm to estimateLogClay and LogSilt, respectively.

Keywords

• Partial Least Squares Regression (PLSR)
• Principal Component Regression (PCR)
• Spectroscopy

20.1001.1.20084757.1397.32.1.6.0

References

1- Araújo S.R., Demattê J.A.M., Franceschini M.H.D., Rizzo R., Stenberg B., and Wetterlind J. 2013. Improving the predictive performance of a national vis-NIR spectroscopic library by comparing clustering data transformation, and data-mining calibration techniques. p. 2431-2440. Anais XVI SimposioBrasileiro de SensoriamentoRemoto, 13 - 18 April. 2013. Foz do Iguaçu, PR, Brazil.

2- Barnes R.J., Dhanoa M.S., and Lister S.J. 1989. Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra. Applied Spectroscopy, 43:772-777.

3- Bellinaso H., Demattê J.A.M., and Araújo S.R. 2010. Soil spectral library and its use in soil classification. Rev. Bras.Ciênc. Solo, 34:861–870.

4- Ben-Dor E., and Banin A. 1995. Near-infrared analysis (NIRA) as a method to simultaneously evaluate spectral featureless constituents in soils. Soil Science, 159:259–270.

5- Bureau S., Ruiz D., Reich M., Gouble B., Bertrand D., Audergon J.M., and Renard C.M.G.C. 2009. Rapid and non-destructive analysis of apricot fruit quality using FT-near-infrared spectroscopy. Food Chemistry, 113: 1323-1328.

6- Cai Y., Cabrera J.C., Georgiadis M., and Jayachandran K. 2002. Assessment of arsenic mobility in the soils of some golf courses in South Florida. Science of the Total Environment, 291:123–134.

7- CAMO AS Press. 2006. The Unscrambler Tutorials. CAMO Software Research & Development Team, CAMO Software, NedreVollgate 8, N-0158, Oslo, Norway, retrieved on March 20/2012 from http://www.camo.com/

8- Chang C.W., Laird D.A., Mausbach M.J., and Hurburgh C.R. 2001. Near-infrared reflectance spectroscopy-principal components regression analyses of soil properties Soil Science Society of America Journal, 65:480–490.

9- Chen L., Sheng-lu Z., Shao-hua W., Qing Z., and Qi D. 2014. Spectral Response of Different Eroded Soils in Subtropical China: A Case Study in Changting County, China. Journal of Materials Science, 11: 697-707.

10- Cho M.A., Skidmore A., Corsi F., van Wieren S.E., and Sobhan I. 2007. Estimation of green grass/herb biomass from airborne hyperspectral imagery using spectral indices and partial least squares regression. International Journal of Applied Earth Observation and Geoinformation, 9: 414-424.

11- Conforti M., Froio R., Matteucci G., and Buttafuoco G. 2015. Visible and near infrared spectroscopy for predicting texture in forest soil: an application in southern Italy. iForest, 8: 339-347

12- Curcio D., Ciraolo G., D’Asaro F., and Minacapillia M. 2013. Prediction of soil texture distributions using VNIR-SWIR reflectance spectroscopy. Procedia Environmental Sciences, 19:494 – 503.

13- Divya Y., Sanjeevi S., and Ilamparuthi K. 2014. A study on the hyperspectral signatures of sandy soils with varying texture and water content. Arabian Journal of Geosciences, 7:3537-3545.

14- Gholizadeh A., Boruvkai L., Saberioon M.M., Kozaki J., Vasati R., and Nemeki K. 2015. Comparing Different Data Preprocessing. Soil & Water Res., 10:218–227.

15- Gomez C., Rossel V., and McBratney B. 2008. Soil organic carbon prediction by hyperspectral remote sensing and field vis-NIR spectroscopy: An Australian case study. Geoderma, 146:403–411.

16- Hillel D. Applications of Soil Physics. Academic Press Inc. 1980.

17- Hunt G.R., and Ashley R.P. 1979. Spectra of altered rocks in the visible and near infrared. Economic Geology, 74:1613-1629.

18- Jahanshahi R., and Zare, M, 2015. Assessment of heavy metals pollution in groundwater of Golgohar iron ore mine area, Iran. Environmental Earth Science, 74:505–520.

19- Ji J.F., Balsam W.L., Chen J., and Liu L.W. 2002. Rapid and quantitative measurement of hematite and goethite in the Chinese Loess-Paleosol sequence by diffuse reflectance spectroscopy. Clay Mineral, 50:208–216.

20- Jones H.G., and Vaughan R.A. 2010. Remote sensing of vegetation (principles, techniques, and applications. Oxford University Press, New York.

21- Kaletia A.L., Tian L.F., and Hirschi M.C. 2005. Relationship between soil moisture content and soil surface reflectance. Transactions of the ASAE, 48:1979–1986.

22- Klement A., Jaksik O., Kodesova R., Drabek O., and Boruvka L. 2013. Application of VNIR diffuse reflectance spectroscopy to estimate soil organic carbon content, and content of different forms of iron and manganese. Geophysical Research Abstracts, 15:10846-1.

23- Lacerda M.P.C., Demattê J.A.M., Sato M.V., Fongaro C.T., Gallo B.C., and Souza A.B. 2016. Tropical Texture Determination by Proximal Sensing Using a Regional Spectral Library and Its Relationship with Soil Classification. Remote Sensing, 8(701):1-20.

24- Lagacherie P., Baret F., Feret J.B., Netto J.M., and Robbez-Masson J.M. 2008. Estimation of soil clay and calcium carbonate using laboratory, field and airborne hyperspectral measurements. Remote Sensing of Environment, 112:825–835.

25- Lee K.S., Lee D.H., Sudduth K.A., Chung S.O., Kitchen N.R., Drummond S.T. 2009. Wavelength identification and diffuse reflectance estimation for surface and profile soil properties. American Society of Agricultural and Biological Engineers, 52:683-695

26- Liu Y., Jiang Q., Fei T., Wang J., Shi T., Guo K., Li X., and Chen Y. 2014. Transferability of a Visible and Near-Infrared Model for Soil Organic Matter Estimation in Riparian Landscapes. Remote Sensing, 6:4305-4322

27- Lucas Y., and Gagelli J. 2002. Hyperspectral detection of sand. p. 12-17. 7th International Conference on Remote Sensing for Marine and Coastal Environments, 20-22 May. 2002. Miami, Florida. U.S

28- Luo X., Yu S., and Li X. 2011. Distribution, availability, and sources of trace metals in different particle size fractions of urban soils in Hong Kong: Implications for assessing the risk to human health. Environmental Pollution, 159:1317-1326

29- Martens H., Jensen S.A., Geladi P. 1983. Multivariate linearity transformations for near infrared reflectance spectroscopy.In: Christie OHJ. Proc. Nordic Symp. Stokkland Forlag, Norway: Applied Statistics; p. 205–234.

30- Martens H., Martens M. 2000. Modified jack-knife estimation of parameter uncertainty in bilinear modelling by partial least squares regression (plsr). Food Qual Prefer, 11:5-16.

31- Martens H., Næs T. 1989. Multivariate Calibration. John Wiley & Sons: Chichester, United Kingdom.

32- McCoy R.M. 2005. Field Methods in Remote Sensing. A Division of Guilford Publications, Inc. Spring, New York, U.S, pp. 67-87.

33- McDowell M.L., Bruland G.L., Deenik J.L., Grunwald S., and Knox N.M. 2012. Soil total carbon analysis in Hawaiian soils with visible, near-infrared and mid-infrared diffuse reflectance spectroscopy. Geoderma, 189:312-320

34- Miller R.H., and Keeny D.R. 1992. Methods of Soil Analysis. p. 65-98. Part 1, 2, Physical, Chemical and Mineralogical properties. Soil Science Society of America, Madison, Wisconsin, USA.

35- Mouazen A.M., Kuang B., De Baerdemaeker J., and Ramon H. 2010. Comparison between principal components, partial least squares and artificial neural network analyses for accuracy of measurement of selected soil properties with visible and near infrared 455 spectroscopy. Geoderma, 158:23-31.

36- Nørgaard L., Saudland A., Wagner J., Nielsen J.P., Munck L., and Engelsen S.B. 2000. Interval partial least-squares regression (iPLS): A comparative chemometric study with an example from near-infrared spectroscopy, Applied Spectroscopy, 54:413-419.

37- Qiu H. 2010. Studies on the potential ecological risk and homology correlation of heavy metal in the surface soil. Journal of Agricultural Science, 2:1916-9760.

38- R Development Core Team. 2009. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org/.

39- Rinnan A., Berg F.V.D., and Engelsen S.B. 2009. Review of the most common pre-processing techniques for near-infrared spectra. Trends in Analytical Chemistry, 28: 1201-1222.

40- Schwartz G., Eshel G., and Ben-Dor E. 2011. Reflectance spectroscopy as a tool for monitoring contaminated soils. p. 67-90. In S. Pascucci (ed.) Soil Contamination, In Technology.

41- Sharma S., Goodarzi M., Ramon H., and Saeys W. 2014. Performance evaluation of preprocessing techniques utilizing expert information in multivariate calibration. Talanta, 121:105-12.

42- Shiferaw A., and Hergarten Ch. 2014. Visible near infra-red (VisNIR) spectroscopy for predicting soil organic carbon in Ethiopia.Journal of Ecology and the Natural Environment, 6:126-139.

43- Silva E. B., ten Caten, Dalmolin R.S.D., Dotto A.C., Silva W.C., and Giasson E. 2016. Estimating Soil Texture from a Limited Region of the Visible/Near-Infrared Spectrum..p. 73–87. In A.E. Hartemink and B. Minasny (eds). Digital Soil Morphometr. Springer International Publishing, Switzerland.

44- Stoner E.R., Baumgardner M.F. 1981. Characteristics variations in reflectance of surface soils. Soil Science Society of America Journal, 45: 1161–1165.

45- Summers D., Lewis M., Ostendorf B., and Chittleborough D. 2011. Visible near-infrared reflectance spectroscopy as a predictive indicator of soil properties. Ecological Indicators, 11:123-131.

46- ViscaraRossel R., and Behrens T. 2010. Using data mining to model and interpret soil diffuse reflectance spectra. Geoderma, 158:46-54.

47- Viscarra Rossel R.A., Taylor H.J., and McBratney A.B. 2007. Multivariate calibration of hyperspectral γ-ray energy spectra for proximal soil sensing. European Journal of Soil Science, 58:343-353.

48- Wilding L. 1985. Spatial variability. Its documentation, accommodation, and implication to soil surveys. In D.R. Nielson and J. Bouma (eds) Soil Variability, Pudo, Wagenigen, the Netherlands.

49- Wold S., Martens H., and Wold H. 1983. The multivariate calibration problem in chemistry solved by the PLS method. Matrix Pencils.

50- Wold S., Sjöström M., and Eriksson L. 2001. PLS-regression: A basic tool of chemometrics. Chemometrics and Intelligent Laboratory Systems, 58:109–130.