Fig. 1. The schematic of a NeuroFuzzy System architecture.
Fig. 2. Implementation of fuzzy rules.
1. Background Fuzzy logic has been applied very successfully in many areas where conventional model based approaches are difficult or not cost-effective to implement. However, as system complexity increases, reliable fuzzy rules and membership functions used to describe the system behavior are difficult to determine. Furthermore, due to the dynamic nature of economic and financial applications, rules and membership functions must be adaptive to the changing environment in order to continue to be useful. The main advantage of using neural networks in economic or financial modeling is that they can be synthesized without making use of the detailed, explicit knowledge of the underlying process. However, limited or noisy training data may result in inconsistent, meaningless output. This has been known to be a severe problem of neural networks. Because of their complementary nature, these two technologies can be integrated in a number of ways to overcome the drawbacks of each other. A NeuroFuzzy hybrid system is described in this paper for the purpose of explanation. In the following paragraphs, the term "antecedent" refers to the proposition following "IF" in a fuzzy rule "IF X1 is A1 and X2 is A2 ... THEN Y is B", and that following "THEN" is called the consequent. Membership functions are defined to quantify propositions such as "X is A" and "Y is B". 2. A NeuroFuzzy Hybrid System NeuroFuzzy hybrid systems combine the advantages of fuzzy systems, which deal with explicit knowledge which can be explained and understood, and neural networks which deal with implicit knowledge which can be acquired by learning. Neural network learning provides a good way to adjust the expert's knowledge and automatically generate additional fuzzy rules and membership functions, to meet certain specifications and reduce design time and costs. On the other hand, fuzzy logic enhances the generalization capability of a neural network system by providing more reliable output when extrapolation is needed beyond the limits of the training data. 2.1. NeuroFuzzy Architecture The NeuroFuzzy system consists of the various components of a traditional fuzzy system (For example, see [2]), except that each stage is performed by a layer of hidden neurons, and neural network learning capability is provided to enhance the system knowledge.Fuzzification Layer Each neuron in this layer represents an input membership function of the antecedent of a fuzzy rule. One common method to implement this layer is to express membership functions as discrete points. Thus for a fuzzy rule "IF X1 is A! and X2 is A2 ... THEN Y is B", A's characterize the possibility distribution of the antecedent clause "X is A". Each of the hidden nodes is defined as a fuzzy reference point in the input space. This method can approximate many continuous functions and the degree of error depends very much on the number of discrete points used. Another much better approach is to use a combination of one to two sigmoidal functions and a linear function to represent each membership function in the Fuzzication and Defuzzification layers. The parameters of these neurons can be trained to fine tune the final shape and location of the membership functions. In most designs, the number of neurons in this layer is fixed, but it is possible to add or remove these neurons during training, according to the outputs produced on the training samples. In Fig.2, the membership grade indicating the certainty of "X1 is High" is 0.6, "X1 is Medium" is 0.4, and "X1 is Low" is 0.0. The output of these membership function neurons are connected to the Fuzzy Rule Layer as specified by the fuzzy rules, using links with fixed weights of unity. In Fig.2, each small rectangle represents a neuron.Fuzzy Rule LayerThis layer represents the fuzzy rule base and its function is to perform the fuzzy logical operations. Each neuron represents a fuzzy rule such as "IF X1 is A1 and X2 is A2 ... THEN Y is B" and it calculates the certainty of each compound proposition "IF X1 is A1 and X2 is A2 ... " which indicates the goodness of fit, that is, how well the prerequisites of each fuzzy rule are satisfied. The neurons have a linear function, and their output are connected to the Defuzzification Layer by weighted links. The weights of these links represent the relative significance of the rules associated with the neurons. Their values can be preset according to the expert or initialized to be 1.0, and then trained to reflect their actual importance to the output membership functions contained in the Defuzzification Layer. Defuzzification Layer The function of this layer is for rule evaluation. Each neuron in this layer represents a consequent proposition "THEN Y is B" and its membership function can be implemented by combining one or two sigmoidal functions and linear functions. The certainty of each consequent proposition is calculated, and is regarded as the goodness of fit of those fuzzy rules which have the same consequent proposition. The weight of each output link from these neurons represents the center of gravity of each output membership function of the consequent, and is trainable. The final output value is then calculated using the center of gravity method. 2.2. Fuzzy Rule Implementation In the NeuroFuzzy system illustrated in Fig. 2, a set of trading rules such as: Rule 1: IF Prime_Rate is HIGH and DJIA is HIGH THEN Dollars_Purchase is LOW Rule 2: IF Prime_Rate is LOW and DJIA is LOW THEN Dollars_Purchase is HIGH can be programmed easily. A more general format is given below: Rule 1: IF X1 is High and X2 is Low THEN Y is High .................. 0.8 Rule 2: IF X1 is Medium and X2 is High THEN Y is Medium ....0.5 The value at the end of each rule represents the initial weight of the rule, and will be adjusted to its appropriate level at the end of training. If all the rules have the same subject "Y" for the consequent propositions, then only 1 output node is needed. Fig. 2 illustrates a system with three subjects "Y1", "Y2" and "Y3". The output values could be Dollars transacted, credit worthiness, valuation of real estates, market indices, buy/sell signals and etc. 3. Training of NeuroFuzzy System The structure in Fig. 2 can be configured with initial values specified by human experts, and then further tuned by using a training algorithm such as backpropagation as follows: Step 1: Present an input data sample, compute the corresponding output Step 2: Compute the error between the output(s) and the actual target(s) Step 3: The connection weights and membership functions are adjusted Step 4: At a fixed number of epochs, delete useless rule and membership function nodes, and add in new ones Step 5: IF Error > Tolerance THEN goto Step 1 ELSE stop. When the error level drops to below the user-specified tolerance, the final interconnection weights reflect the changes in the initial fuzzy rules and membership functions. If the resulting weight of a rule is close to zero, the rule can be safely removed from the rule base, since it is insignificant compared to others. Also, the shape and position of the membership functions in the Fuzzification and Defuzzification Layers can be fine tuned by adjusting the parameters of the neurons in these layers, during the training process. 4. Conclusion NeuroFuzzy systems offer the precision and learning capability of neural networks, and yet are easy to understand like fuzzy systems. Explicit knowledge acquired from experts can be easily incorporated into such a system, and implicit knowledge can be learned from training samples to enhance the accuracy of the output. Furthermore, the modified and new rules can be extracted from a properly trained NeuroFuzzy system, to explain how the results are derived. There are also many other ways to combine neural and fuzzy techniques, to improve the learning speed, adjust learning and momentum rates, etc. Also, newer technologies such as genetic algorithm can be integrated to further enhance the performance of the hybrid systems.
Copyright © 1993-96, NIBS Pte Ltd.