IDOL - Adding New Engines Into IDOL 12.x. Cluster
Posted by Kimmo Pyhältö (M-Files) on 20 December 2019 10:13 AM
IDOL index can be expanded after the initial deployment by adding extra engines to the cluster. Currently, up to 16 engines can be used in a single cluster meaning some 20 million documents per index. Even adding the engine is quite straightforward operation, the vault has to be re-indexed (only .idx indexing) after the addition. As an extra motivation, the index backup is created during the process.
This document describes how to add new engines into existing cluster and how to re-index it so IDOL can utilize the extra capacity.
NOTE 1! This document is valid for IDOL version 12.x and beyond.
Keywords: IDOL, cluster, engine, addition
- Existing IDOL 12.x cluster
2. Steps to add new engines into the existing cluster
1. During the initial installation you have used IDOL12_setup_tool.xlsm to plan the cluster and to generate the installation scripts. You have also saved that file to have the current configuration for future usage. Now is that time. Open the file.
2. Select the last content engine line (whole line) and copy it.
Result: The line is copied under the last content engine and the conflict appears due to the same port numbers.
6. Generate scripts.
Result: A new installation scripts have been generated to the installation folder. The notable one is that has SOLO_INSTALL on it.
Result: If the script is successful, it will tell few manual steps you have to do to make the cluster aware of the new engine.
9. Add new engines to the DIH's and DAH's configuration files.
Example: We have added two extra engines using port ranges 13000 and 14000. In following lines, changes are in orange color and added lines are in blue color. The usual location of dih.cfg and dah.cfg is C:\IDOL12\bin\[installation name]\dih and C:\IDOL12\bin\[installation name]\dah.
10. Restart DIH and DAH services.
11. Now it is time to export the index and import it again so the DIH will more or less spread the contents of the index to the new engines. Refer to IDOL 12 installation instructions chapters 10.1 and 10.2 for exporting and importing (= Re-indexing) the data.
3. Process in the future
The process of adding new engines will become easier in the future, when we abandon the idea of "fixed" index content. That will make the need of re-indexing obsolete.