Hostname: page-component-cd9895bd7-7cvxr Total loading time: 0 Render date: 2024-12-26T17:55:11.641Z Has data issue: false hasContentIssue false
  • English
  • Français

High Throughput Quantitative Metallography for Complex Microstructures Using Deep Learning: A Case Study in Ultrahigh Carbon Steel

Published online by Cambridge University Press:  14 March 2019

Brian L. DeCost Open the ORCID record for Brian L. DeCost [Opens in a new window] ,
Bo Lei Open the ORCID record for Bo Lei [Opens in a new window] ,
Toby Francis Open the ORCID record for Toby Francis [Opens in a new window]  and
Elizabeth A. Holm Open the ORCID record for Elizabeth A. Holm [Opens in a new window]
Brian L. DeCost*
Affiliation:
Material Measurement Laboratory, National Institute of Standards and Technology, Gaithersburg MD, 20899, USA
Bo Lei
Affiliation:
Materials Science and Engineering, Carnegie Mellon University, Pittsburgh PA, 15213, USA
Toby Francis
Affiliation:
Materials Science and Engineering, Carnegie Mellon University, Pittsburgh PA, 15213, USA
Elizabeth A. Holm
Affiliation:
Materials Science and Engineering, Carnegie Mellon University, Pittsburgh PA, 15213, USA
*
*Author for correspondence: Brian L. DeCost, E-mail: brian.decost@nist.gov
Get access

Abstract

We apply a deep convolutional neural network segmentation model to enable novel automated microstructure segmentation applications for complex microstructures typically evaluated manually and subjectively. We explore two microstructure segmentation tasks in an openly available ultrahigh carbon steel microstructure dataset: segmenting cementite particles in the spheroidized matrix, and segmenting larger fields of view featuring grain boundary carbide, spheroidized particle matrix, particle-free grain boundary denuded zone, and Widmanstätten cementite. We also demonstrate how to combine these data-driven microstructure segmentation models to obtain empirical cementite particle size and denuded zone width distributions from more complex micrographs containing multiple microconstituents. The full annotated dataset is available on materialsdata.nist.gov.

Keywords

deep learningmicrostructuresegmentationsteel
Type
Materials Science Applications
Information
Microscopy and Microanalysis , Volume 25 , Issue 1 , February 2019 , pp. 21 - 29
Copyright
Copyright © Microscopy Society of America 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Azimi, SM, Britz, D, Engstler, M, Fritz, M & Mücklich, F (2018). Advanced steel microstructural classification by deep learning methods. Sci Rep 8, 2128. Available at http://dx.doi.org/10.1038/s41598-018-20037-5.Google Scholar
Badrinarayanan, V, Kendall, A & Cipolla, R (2017). Segnet: a deep convolutional encoder-decoder architecture for scene segmentation. IEEE Trans Pattern Anal Mach Intell 39, pp. 1–1.10.1109/TPAMI.2016.2644615Google Scholar
Bansal, A, Chen, X, Russell, B, Gupta, A & Ramanan, D (2017). PixelNet: Representation of the pixels, by the pixels, and for the pixels, CoRR. Available at http://arxiv.org/abs/1702.06506v1.Google Scholar
Bansal, A, Russell, B & Gupta, A (2016). Marr revisited: 2d–3d alignment via surface normal prediction. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 5965–5974.Google Scholar
Canny, J (1987). A computational approach to edge detection, Readings in Computer Vision 184–203. Available at http://dx.doi.org/10.1016/b978-0-08-051581-6.50024-6.Google Scholar
Chollet, F (2015). Keras. Available at https://github.com/fchollet/keras.Google Scholar
Chowdhury, A, Kautz, E, Yener, B & Lewis, D (2016). Image driven machine learning methods for microstructure recognition. Comput Mater Sci 123, 176–187.Google Scholar
Decost, BL, Francis, T & Holm, EA (2017 a). Exploring the microstructure manifold: image texture representations applied to ultrahigh carbon steel microstructures. Acta Mater 133, 30–40. Available at http://arxiv.org/abs/1702.01117.Google Scholar
Decost, BL, Hecht, MD, Francis, T, Webler, BA, Picard, YN & Holm, EA (2017 b). UHCSDB (ultrahigh carbon steel micrograph database): tools for exploring large heterogeneous microstructure datasets. Integrating Materials and Manufacturing Innovation 6, 197–205. Available at http://uhcsdb.materials.cmu.edu.Google Scholar
Decost, BL, Hecht, MD, Sibley, T, Francis, T, Webler, BA, Picard, Y & Holm, EA (2018). Ultrahigh carbon steel micrographs: Microconstituent annotations. Available at https://hdl.handle.net/11256/964.Google Scholar
Decost, BL & Holm, EA (2015). A computer vision approach for automated analysis and classification of microstructural image data. Comput Mater Sci 110, 126–133.10.1016/j.commatsci.2015.08.011Google Scholar
Goodfellow, I, Bengio, Y & Courville, A (2016). Deep Learning, Cambridge, MA: MIT Press. Available at http://www.deeplearningbook.org.Google Scholar
Hall, E (1951). The deformation and ageing of mild steel: III discussion of results. Proceedings of the Physical Society Section B 64, 747.10.1088/0370-1301/64/9/303Google Scholar
Hariharan, B, Arbelaez, P, Girshick, R & Malik, J (2015). Hypercolumns for object segmentation and fine-grained localization. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
Hastie, T, Friedman, J. & Tibshirani, R (2001). The elements of statistical learning, Springer Series in Statistics. Available at http://dx.doi.org/10.1007/978-0-387-21606-5.Google Scholar
He, K, Zhang, X, Ren, S & Sun, J (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. 2015 IEEE International Conference on Computer Vision (ICCV). Available at http://dx.doi.org/10.1109/iccv.2015.123.Google Scholar
Hecht, MD, Decost, BL, Francis, T, Holm, EA, Picard, YN & Webler, BA (2017 a). Ultrahigh carbon steel micrographs. Available at https://hdl.handle.net/11256/940.Google Scholar
Hecht, MD, Picard, YN & Webler, BA (2017 b). Coarsening of inter and intragranular proeutectoid cementite in an initially pearlitic 2C-4Cr ultrahigh carbon steel. Metallurgical and Materials Transactions A 48, 1–16.Google Scholar
Hecht, MD, Webler, BA & Picard, YN (2016). Digital image analysis to quantify carbide networks in ultrahigh carbon steels. Mater Charact 117, 134–143.Google Scholar
Ioffe, S & Szegedy, C (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 37, Bach, F & Blei, D (Eds.), pp. 448–456, Cambridge, MA: PMLR. Available at http://proceedings.mlr.press/v37/ioffe15.html.Google Scholar
Jégou, S, Drozdzal, M, Vazquez, D, Romero, A & Bengio, Y (2016). The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation, CoRR. Available at http://arxiv.org/abs/1611.09326v2.Google Scholar
Kendall, A, Badrinarayanan, V & Cipolla, R (2015). Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding, CoRR. Available at http://arxiv.org/abs/1511.02680v2.Google Scholar
Kingma, DP & Ba, J (2014). Adam: A method for stochastic optimization, CoRR. Available at http://arxiv.org/abs/1412.6980v9.Google Scholar
Lecun, Y, Bengio, Y & Hinton, G (2015). Deep learning. Nature 521, 436–444. Available at http://dx.doi.org/10.1038/nature14539.Google Scholar
Lin, TY, Goyal, P, Girshick, R, He, K & Dollár, P (2017). Focal loss for dense object detection, CoRR. Available at http://arxiv.org/abs/1708.02002v2.Google Scholar
Long, J, Shelhamer, E & Darrell, T (2015). Fully convolutional networks for semantic segmentation, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Available at http://dx.doi.org/10.1109/cvpr.2015.7298965.Google Scholar
Loshchilov, I & Hutter, F (2017). Fixing weight decay regularization in adam, CoRR. Available at http://arxiv.org/abs/1711.05101v2.Google Scholar
Lubbers, N, Lookman, T & Barros, K (2016). Inferring low-dimensional microstructure representations using convolutional neural networks, CoRR. Available at http://arxiv.org/abs/1611.02764v1.Google Scholar
Massey, FJ (1951). The Kolmogorov–Smirnov test for goodness of fit. J Am Stat Assoc 46, 68–78. Available at http://dx.doi.org/10.1080/01621459.1951.10500769.Google Scholar
Nair, V & Hinton, GE (2010). Rectified linear units improve restricted Boltzmann machines. Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML'10, 807–814, Omnipress. Available at http://dl.acm.org/citation.cfm?id=3104322.3104425.Google Scholar
Nolden, M, Zelzer, S, Seitel, A, Wald, D, Müller, M, Franz, AM, Maleike, D, Fangerau, M, Baumhauer, M, Maier-Hein, L, Maier-Hein, KH, Meinzer, HP & Wolf, I (2013). The medical imaging interaction toolkit: Challenges and advances. Int J Comput Assist Radiol Surg 8, 607–620. Available at https://doi.org/10.1007/s11548-013-0840-8.Google Scholar
Otsu, N (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics 9, 62–66. Available at http://dx.doi.org/10.1109/tsmc.1979.4310076.Google Scholar
Pedregosa, F, Varoquaux, G, Gramfort, A, Michel, V, Thirion, B, Grisel, O, Blondel, M, Prettenhofer, P, Weiss, R, Dubourg, V, Vanderplas, J, Passos, A, Cournapeau, D, Brucher, M, Perrot, M & Duchesnay, E (2011). Scikit-learn: Machine learning in python. The Journal of Machine Learning Research 12, 2825–2830.Google Scholar
Ronneberger, O, Fischer, P & Brox, T (2015). U-net: Convolutional networks for biomedical image segmentation, Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015 234–241. Available at http://dx.doi.org/10.1007/978-3-319-24574-4_28.Google Scholar
Russakovsky, O, Deng, J, Su, H, Krause, J, Satheesh, S, Ma, S, Huang, Z, Karpathy, A, Khosla, A, Bernstein, M, Berg, AC, & Li, FF (2015). Imagenet large scale visual recognition challenge. Int J Comput Vision 115, 211–252. Available at http://dx.doi.org/10.1007/s11263-015-0816-y.Google Scholar
Simonyan, K & Zisserman, A (2014). Very deep convolutional networks for large-scale image recognition, CoRR abs/1409.1556. Available at http://arxiv.org/abs/1409.1556.Google Scholar
Srivastava, N, Hinton, GE, Krizhevsky, A, Sutskever, I & Salakhutdinov, R (2014). Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15, 1929–1958. Available at http://dl.acm.org/citation.cfm?id=2670313.Google Scholar
Van Der Walt, S, Schönberger, JL, Nunez-Iglesias, J, Boulogne, F, Warner, JD, Yager, N, Gouillart, E & Yu, T (2014). Scikit-image: Image processing in python. Peer J 2, e453. Available at https://doi.org/10.7717/peerj.453.Google Scholar
Vincent, L & Soille, P (1991). Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Trans Pattern Anal Mach Intell 13, 583–598. Available at http://dx.doi.org/10.1109/34.87344.Google Scholar
Wang, P, Chen, P, Yuan, Y, Liu, D, Huang, Z, Hou, X & Cottrell, G (2017). Understanding convolution for semantic segmentation, CoRR. Available at http://arxiv.org/abs/1702.08502v1.Google Scholar
Zener, C (1948). Quoted by CS Smith. Trans AIME 175, 15, quoted by CS Smith.Google Scholar
Supplementary material: PDF

DeCost et al. supplementary material

DeCost et al. supplementary material 1

Download DeCost et al. supplementary material(PDF)
PDF 114.4 MB
Supplementary material: PDF

DeCost et al. supplementary material

DeCost et al. supplementary material 2

Download DeCost et al. supplementary material(PDF)
PDF 2.1 MB
Supplementary material: PDF

DeCost et al. supplementary material

DeCost et al. supplementary material 3

Download DeCost et al. supplementary material(PDF)
PDF 2.8 MB