Investigation of the Use of an Artificial

Neural Network Method for the Prediction of

Crystal Structures of Zeolites from XRD Data

Melkon Tatlier

and H. Kerem Cigizoglu

Department of Chemical Engineering, Istanbul Technical University

Maslak, 34469 Istanbul, Turkey

Department of Civil Engineering, Division of Hydraulics, Istanbul Technical University

Maslak, 34469 Istanbul, Turkey

Abstract. The possibility of using artificial neural network methods for the

prediction of zeolite crystal structures, such as pore size and unit cell dimen-

sions, from X-ray diffraction patterns was investigated. The Generalized Re-

gression Neural Network method and X-ray diffraction data obtained from lite-

rature were utilized in these investigations. The predictions made by using this

neural network method were, in general, more reliable than those performed by

regression. The best predictions were achieved for the estimation of the pore

size, while the neural network method improved significantly the very poor re-

sults obtained by regression for the unit cell dimensions.

1 Introduction

Zeolites are hydrated microporous crystalline aluminosilicates that may be used in

diverse applications related to ion exchange, catalysis, adsorption and separation [1-

3]. Zeolites have developed into a large industry due to their unique and versatile

properties. They may be utilized in the separation of linear and branched hydrocar-

bons, for catalytic cracking and hydrocracking or as detergent builders, to name a

few, while a significant number of potential applications are waiting to emerge. Zeo-

lites may occur naturally or be synthesized in laboratory conditions. The most signifi-

cant parameters determining the type of the zeolite formed from a certain initial reac-

tion mixture composition are the synthesis time and temperature as well as the molar

ratios of the reactants. Suitable reagents that form a clear solution or a gel mixture

should be used to obtain different types of zeolites. After carrying out the synthesis

procedure with these reagents, the solid material formed in the solution should be

separated by filtration, which is then characterized by using various techniques. X-ray

diffraction (XRD) is one of the basic and essential techniques to characterize the solid

material thus obtained. XRD may be used to determine the crystallographic structure,

grain size and orientation of the crystals. It is commonly utilized to identify unknown

substances by comparing diffraction data against a database. The relative abundance

of crystalline materials in solid mixtures may also be determined by this technique.

Tatlier M. and Cigizoglu H. (2009).

Investigation of the Use of an Artiﬁcial Neural Network Method for the Prediction of Crystal Structures of Zeolites from XRD Data.

In Proceedings of the 5th International Workshop on Artiﬁcial Neural Networks and Intelligent Information Processing, pages 97-105

DOI: 10.5220/0002263700970105

 SciTePress

Additionally, when coupled with lattice refinement techniques, it can provide

structural information on unknown materials. The achievement of this last deed is not

a simple task to perform and requires significant amount of knowledge on the numer-

ous peaks pertaining to the X-ray diffraction patterns of different materials.

Artificial neural networks (ANNs) have the ability to learn from input data and are

very useful for the prediction of complex high-dimensional data. ANN methods have

a broad range of applications, including research in chemical engineering. Artificial

neural networks have been successfully used for dynamic modeling and control of

chemical processes and fault diagnosis [4], in the catalytic modeling and design of

solid catalysts [5] and for modeling the kinetics of a chemical reaction [6]. The appli-

cability of ANN methods in emulsion liquid membranes [7] and in the predic-

tion/estimation of the vapor-liquid equilibrium data [8] has been investigated. It has

also been shown that ANN methods might learn efficiently from available zeolite

synthesis data in the literature to predict the complex relationship between the chemi-

cal compositions of initial reaction mixtures and the zeolites formed from them [9].

A detailed theoretical investigation of the rather complex and high dimensional re-

lationship between the XRD peaks and the crystallographic properties of various

zeolites (as well as other crystalline materials) may be very useful to provide a more

common and practical use of the XRD technique in the prediction of the crystal struc-

tures of unknown zeolites and other materials.

In this study, the Generalized Regres-

sion Neural Network (GRNN) method was utilized to perform this investigation. The

results obtained were compared to XRD data reported in the literature, as well as to

the estimations made by using multilinear regression.

2 Theory

2.1 X-Ray Diffraction Technique

Crystals are regular arrays of atoms, and X-rays can be considered as waves of

electromagnetic radiation. Atoms scatter X-ray waves, primarily through their

electrons. An X-ray striking an electron produces secondary spherical waves

emanating from the electron, which is known as elastic scattering. Although these

waves cancel one another out in most directions through destructive interference, they

add constructively in a few specific directions, determined by Bragg’s law,

2dsinθ = nλ

(1)

where d is the spacing between diffracting planes in Å, θ is the incident angle in

degrees, n is any integer, and λ is the wavelength of the beam in Å. These specific

directions appear as spots on the diffraction pattern. It should be mentioned that X-

rays have wavelengths on the order of a few angstroms, the same as typical

interatomic distances in crystalline solids. This means that X-rays can be diffracted

from minerals which, by definition, are crystalline and have regularly repeating

atomic structures.

In the XRD technique, the X-ray intensity is recorded and reported

as a function of the 2θ angle.

2.2 Artificial Neural Networks

Artificial neural networks are black box models that can perform an estimation using

limited input and output data patterns. In this study, the Generalized Regression

Neural Network (GRNN) method was used to relate the XRD data to the properties of

the crystal structures of zeolites.

The basics of the GRNN can be found in the literature [10,11].

The GRNN method

does not require an iterative training procedure but instead estimates any arbitrary

function between input and output vectors, drawing the function estimate directly

from the training data. This method is consistent, that is, as the training set size be-

comes large, the estimation error approaches zero, with only mild restrictions on the

function. The GRNN is used for estimation of continuous variables, as in standard

regression techniques. It is based on a standard statistical technique called kernel

regression. By definition, the regression of a dependent variable y on an independent

x estimates the most probable value for y, given x and a training set. The regression

method will produce the estimated value of y, which minimizes the mean-squared

error. The GRNN consists of four layers: input layer, pattern layer, summation layer,

and output layer. The first layer is fully connected to the second, pattern layer, where

each unit represents a training pattern and its output is a measure of the distance of

the input from the stored patterns. Each pattern layer unit is connected to the two

neurons in the summation layer: S-summation neuron and D-summation neuron. The

S-summation neuron computes the sum of the weighted outputs of the pattern layer

while the D-summation neuron calculates the unweighted outputs of the pattern neu-

rons. The connection weight between the i

neuron in the pattern layer and the S-

summation neuron is y

, the target output value corresponding to the i

input pattern.

For D-summation neuron, the connection weight is unity. The output layer merely

divides the output of each S-summation neuron by that of each D-summation neuron.

In this method, the spread σ is a smoothing parameter, the optimal value of which is

often determined experimentally [12]. When the spread parameter σ is made large,

the estimated density is forced to be smooth and in the limit becomes a multivariate

Gaussian with covariance σ

I. On the other hand, a smaller value of σ allows the

estimated density to assume non-Gaussian shapes, but with the hazard that wild

points may have too great an effect on the estimate. In this study, different spreads

were tried to find the best one that gave the minimum difference between predicted

and experimental values for the utilization of the cross-validation data.

2.3 Method

Zeolites are hydrated microporous crystalline materials. The zeolite framework con-

sists of an assemblage of SiO

and AlO

tetrahedra, joined together in various regular

arrangements through shared oxygen atoms, to form an open crystal lattice. The mi-

cropore structure is determined by the crystal lattice, which contains pores of molecu-

lar dimensions into which guest molecules can penetrate. The cations (e.g., Na) are

placed in special positions near the Al atoms. The pore size varies for different zeo-

lites, depending on the arrangement of the atoms forming the zeolite crystal structure.

The crystal structure of a material or the arrangement of atoms in a crystal structure

can be described in terms of its unit cell. The unit cell is a tiny box with one or more

spatial arrangements of atoms. The unit cells stacked in three-dimensional space

describe the bulk arrangement of atoms of the crystal. The crystal structure has a

three dimensional shape. The unit cell may be represented by its lattice parameters,

including the length of the cell edges and the angles between them.

Data obtained from the literature [13], describing the XRD patterns of different

zeolites were used in the estimations carried out by using the ANN method. Prelimi-

nary estimations indicated that the GRNN method was more successful in the predic-

tion of the zeolite crystal structure from XRD data when compared to the Radial Ba-

sis Function-Based Neural Networks (RBF) and Feed Forward Back Propagation

(FFBP) methods, which were also examined. Thus, the GRNN method was used for

the estimations performed in detail. The components of the input vector were the 2θ

angles of eight XRD peaks with the highest intensity pertaining to different zeolites,

while the components of the output vector were the pore sizes (r) and lengths of the

unit cell edges (a,b,c) of these zeolites. The pore sizes and unit cell dimensions of

zeolites generally vary between a few angstroms and a few nanometers. Since the

pores of some zeolites are not uniform and some others may have pore channels of

different lengths, the largest dimensions of the pores were taken into consideration in

this study.

The application of the ANN to data consisted of two steps. The first step was the

training of the neural networks, which comprised the presentation of training data

(data set 1) describing the input and output to the network and obtaining the inter-

connection weights. The components of the input vector were eight different 2θ an-

gles of the XRD peaks, while the components of the output vector were the pore sizes

and the three different lengths of the unit cell edges of corresponding zeolites. The

input and output data were normalized between 0 and 1 prior to the training. Once the

training stage was completed, the ANNs were applied to the cross-validation data

(data set 2). Determining an appropriate architecture of a neural network for a par-

ticular problem is an important issue, since the network topology directly affects its

computational complexity and its generalization capability. The number of hidden

layers and the number of nodes in the hidden layers were determined after trying

various network structures. The network structure providing the best result was de-

termined according to the success of the predictions performed by using the cross-

validation data set. The ANN method was used to predict only one component of the

output vector at a time.

The number of data used for training was 55 while that used for cross-validation

was 7. The zeolites consisting of silicon, aluminum, oxygen, water and different ca-

tions were taken into consideration in the investigations carried out in this study.

The results obtained by using the GRNN method were compared to the actual val-

ues [13] as well as to those values estimated by using multilinear regression. In re-

gression, the relationships between the 2θ angles of eight XRD peaks and the pore

sizes and lengths of the unit cell edges of zeolites were determined by using data set

1. The information obtained was used in the estimation of the pore sizes and lengths

of the unit cell edges of the zeolites investigated in data set 2. Since, to our know-

ledge, a similar theoretical attempt, for determining such a relationship has not been

performed before, the comparison of the results obtained from ANN methods to those

determined by multilinear regression may be a reasonable first approach. The regres-

100

sion model tested in this study was of simple linear form, as given below. R

represents either the pore size or the lengths of the unit cell edges of zeolites

R = a

+ a

(2θ

)+ a

(2θ

)+ a

(2θ

)+ a

(2θ

)+ a

(2θ

)+ a

(2θ

)+ a

(2θ

)+ a

(2θ

) a

(2θ

) (2)

The relative error (d) was used to monitor the success of the ANN method and re-

gression used in the prediction of zeolite crystal properties from the 2θ angles of the

X-ray diffraction patterns. d was determined by taking into consideration the devia-

tion (%) of the pore sizes or lengths of the unit cell edges of zeolites, calculated by

using the ANN method (c

calc

), from the corresponding actual values (c

act

d = ⏐c

act

-c

calc

⏐/ c

act

x 100 (3)

was defined as the arithmetic mean of the relative errors obtained for the dif-

ferent data used in prediction.

3 Results and Discussion

3.1 Criteria assuring Best Performance for the GRNN Method

The predictions of the pore sizes and lengths of the unit cell edges of zeolites investi-

gated in data set 2 were performed by using GRNN method mentioned above, and

data set 1 for training. As mentioned before, the network structure providing the best

result was determined according to the success of the predictions performed by using

the cross-validation data. It was also established that the conditions providing the best

results in the testing stage could allow the ANN method to exhibit quite high perfor-

mances in the training stage.

For the GRNN method, spread factors in the range 0.02-0.2 were the conditions

determined to give the best results. The spread factor was determined to be equal to

0.15, 0.075, 0.2 and 0.02 for r (pore size of zeolites), a, b and c (lengths of unit cell

edges of zeolites), respectively, by using the cross-validation data for the optimiza-

tion. When the optimization was performed by using training data (data set 1), with-

out taking into consideration data set 2, the d

values, representing the deviation of

the predicted values of data set 1 from experimental values, were less than 10% for all

the cases investigated. When the spread factor was optimized by using data set 2, the

predictive power of the GRNN method was not reduced significantly. The predictions

made by using the GRNN method for the pore size and lengths of the unit cell edges

of zeolites are detailed below.

3.2 Evaluation of the Predictions made by the GRNN Method

The results obtained by the GRNN method are depicted in Figures 1-4 for r (pore

size), a, b, and c (lengths of the unit cell edges) of zeolites, respectively. The devia-

tions of the results obtained by using the ANN method from the actual values may be

observed more clearly from Table 1. The performance exhibited by multilinear re-

101

gression may also be seen in the table. It may be observed from Figures 1-4 and Table

1 that the GRNN method provided fairly good fits to the actual results for most of the

data, though there were some discrepancies. The average deviation from actual results

was smaller for the pore size predictions while the largest deviation was observed for

length, a, of the unit cell edge.

Table 1. Relative errors obtained for the predictions.

Method d

(%)

r a b c

GRNN 12.0 30.9 17.5 22.2

regression 14.7 149.4 169.2 160.7

012345678

Data number

r (Å)

Fig. 1. Zeolite pore size predictions by (x) GRNN in comparison to (_) actual values.

The results obtained for regression given in Table 1 should also be taken into con-

sideration before arriving at a conclusion about the success of the ANN method in the

prediction of the zeolite crystal structure properties. When multilinear regression was

utilized, the average deviation from the actual values was slightly higher than that of

the GRNN method for the prediction of the pore size. However, the predictions of the

lengths of the unit cell edges were very poor with multilinear regression. The average

deviation was equal to about 150-170% for the estimation of all the three edges of the

unit cell. When this information is taken into consideration, the average deviations

between 17% and 31% provided by the GRNN method may be regarded to be quite

promising. For further improvement, additional XRD data, for example, those per-

taining to zeolite-like materials may also be included for the training of the neural

networks. Furthermore, the height and width of the XRD peaks may also be taken

into consideration as components of the input vector. Different ANN methods may

also be tested for possible improvements in the prediction of crystal structures from

XRD data.

102

The superiority of the ANNs over conventional methods for the prediction of com-

plex and high dimensional relationships, such as the one investigated in this study,

can be attributed to the capability of the ANNs to capture the nonlinear features and

generalize the structure of the whole data set. ANN methods are flexible alternatives

and standard ANN software can be used to construct intricate multipurpose nonlinear

solutions. The method has no limitations in the form of fixed assumptions or formal

constraints. The neural network has a distributed processing structure. Each individu-

al processing unit or the weighted connection between two units is responsible for

one small part of the input–output mapping system.

012345678

Data number

a (Å)

Fig. 2. Unit cell length, a, predictions by (x) GRNN in comparison to (_) actual values.

012345678

Data number

b (Å)

Fig. 3. Unit cell length, b, predictions by (x) GRNN in comparison to (_) actual values.

103

012345678

Data number

c(Å)

Fig. 4. Unit cell length, c, predictions by (x) GRNN in comparison to (_) actual values.

4 Conclusions

It was determined that neural networks might learn from XRD data to predict some

properties of the crystal structures of zeolites. The predictions made were, in general,

much more reliable than those performed by the multilinear regression. The best

prediction was made for the pore sizes of zeolites, which also represented the case

where the difference between the success of the predictions made by regression and

neural networks was the smallest. The improvement provided by the use of the

GRNN method, when compared to regression, was quite significant for the predic-

tions of the lengths of the unit cells of zeolites.

The use of artificial neural network methods may allow a better understanding of

the relationship between the X-ray diffraction patterns and the crystallographic prop-

erties of zeolites as well as other materials. This will ease and support the discovery

of novel crystal materials since a short and practical characterization by using availa-

ble XRD data will become possible. It should also be remembered that in case train-

ing and cross-validation data other than those adopted in this study, (e.g., those per-

taining to different types of materials) could be used with GRNN as well as other

ANN models to make the predictions, the relative success of prediction might still

improve. Recurrent neural networks and optimization of neural network architecture

by using genetic programming are methods that may be tested for providing further

developments.

References

1. Weitkamp, J.: Zeolites and Catalysis. Solid State Ionics 131 (2000) 175-188.

104

2. Caro, J., Noack, M., Kolsch, P., Schafer, R.: Zeolite Membranes-State of Their Develop-

ment and Perspective. Microporous Mesoporous Mater. 38 (2000) 3-24.

3. Ruthven, D.M.: Zeolites as Selective Adsorbents. Chem. Eng. Progr. 84 (1988) 42-50.

4. Hussain, M.A.: Review of the Applications of Neural Networks in Chemical Process Con-

trol- Simulation and Online Implementation. Artif. Int. Eng. 13 (1999) 55-68.

5. Huang, K., Chen, F., Lu, D.: Artificial Neural Network-Aided Design of a Multi-

component Catalyst for Methane Oxidative Coupling. Appl. Catal. A 219 (2001) 61-68.

6. Serra, J.M., Corma, A., Chica, A., Argente, E., Botti, V.: Can Artificial Neural Networks

Help the Experimentation in Catalysis? Cat. Today. 81 (2003) 393-403.

7. Chakraborty, M., Bhattacharya, C., Dutta, S.: Studies on the Applicability of Artificial

Neural Network (ANN) in Emulsion Liquid Membranes. J. Membrane Sci. 220 (2003) 155-

164.

8. Sharma, R., Singhal, D., Ghosh, R., Dwivedi, A.: Potential Applications of Artificial Neur-

al Networks to Thermodynamics: Vapor-Liquid Equilibrium Predictions. Comp. Chem.

Eng. 23 (1999) 385-390.

9. Tatlier, M., Cigizoglu, H.K., Erdem-Şenatalar, A.: Artificial Neural Network Methods for

the Estimation of Zeolite Molar Compositions That Form from Different Reaction

Mixtures. Comp. Chem. Eng. 30 (2005) 137-146.

10. Tatlier, M., Cigizoglu, K.B, Cigizoglu, H.K., Erdem-Şenatalar, A.: Low-Silica Zeolite

Coatings Prepared by Using Predictions from an Artificial Neural Network Method. J. Por-

ous Mater. 15 (2008) 389-395.

11. Specht, D.F.: A General Regression Neural Network. IEEE Trans. Neural Networks 2 (1991)

568-576.

12. Tsoukalas, L. H., Uhrig, R. E.: Fuzzy and Neural Approaches in Engineering. Wiley, New

York (1997).

13. Kim, B., Kim, S., Kim, K.: Modeling of Plasma Etching Using a Generalized Regression

Neural Network. Vacuum 71 (2003) 497-503.

14. Treacy, M.M.J., Higgins, J.B.: Collection of Simulated XRD Powder Patterns for Zeolites.

Elsevier, Amsterdam (2001).

105