04295nam a22006255i 4500 978-3-031-57065-0 DE-He213 20251030112305.0 cr nn 008mamaa 240609s2024 sz | s |||| 0|eng d 9783031570650 978-3-031-57065-0 10.1007/978-3-031-57065-0 doi Q334-342 TA347.A78 UYQ bicssc COM004000 bisacsh UYQ thema Riezler, Stefan. author. aut http://id.loc.gov/vocabulary/relators/aut 78491 Validity, Reliability, and Significance [electronic resource] : Empirical Methods for NLP and Data Science / by Stefan Riezler, Michael Hagmann. 2nd ed. 2024. Cham : Springer Nature Switzerland : Imprint: Springer, 2024. XVII, 168 p. 70 illus., 61 illus. in color. online resource. text txt rdacontent computer c rdamedia online resource cr rdacarrier text file PDF rda Synthesis Lectures on Human Language Technologies, 1947-4059 Preface -- Acknowledgments -- Introduction -- Validity -- Reliability -- Significance -- Worked-Through Example: Analyzing Inferential Reproducibility -- Bibliography. This book introduces empirical methods for machine learning with a special focus on applications in natural language processing (NLP) and data science. The authors present problems of validity, reliability, and significance and provide common solutions based on statistical methodology to solve them. The book focuses on model-based empirical methods where data annotations and model predictions are treated as training data for interpretable probabilistic models from the well-understood families of generalized additive models (GAMs) and linear mixed effects models (LMEMs). Based on the interpretable parameters of the trained GAMs or LMEMs, the book presents model-based statistical tests such as a validity test that allows for the detection of circular features that circumvent learning. Furthermore, the book discusses a reliability coefficient using variance decomposition based on random effect parameters of LMEMs. Lastly, a significance test based on the likelihood ratios of nested LMEMs trained on the performance scores of two machine learning models is shown to naturally allow the inclusion of variations in meta-parameter settings into hypothesis testing, and further facilitates a refined system comparison conditional on properties of input data. The book is self-contained with an appendix on the mathematical background of generalized additive models and linear mixed effects models as well as an accompanying webpage with the related R and Python code to replicate the presented experiments. The second edition also features a new hands-on chapter that illustrates how to use the included tools in practical applications. Artificial intelligence. 58441 Machine learning. 55059 Computer science Mathematics. 64041 Mathematical statistics. 67233 Experimental design. 64006 Natural language processing (Computer science). 65131 Artificial intelligence Data processing. Artificial Intelligence. 58441 Machine Learning. 55059 Probability and Statistics in Computer Science. 67234 Design of Experiments. 74873 Natural Language Processing (NLP). 66164 Data Science. Hagmann, Michael. author. aut http://id.loc.gov/vocabulary/relators/aut 78492 SpringerLink (Online service) Springer Nature eBook Printed edition: 9783031570643 Printed edition: 9783031570667 Printed edition: 9783031570674 Synthesis Lectures on Human Language Technologies, 1947-4059 66859 https://doi.org/10.1007/978-3-031-57065-0 ZDB-2-SXSC 60627 60627