Shell Script Generator - Must Use Technology

Shell Script generator is a tool that automates the process of creating UNIX shell scripts used to run the ETL jobs.This tool enables the user either to generate an entire Control M Shell script / Simple shell script or simply a single Sub – Step for the same. Since the tool uses standard templates (By template, we refer to the initial header format, the structure used for writing the functions including naming conventions and process for calling the functions for stepwise execution), once the templates are assured to be bug free the chances of propagation of bugs into the script are reduced.

Note:

In concerto we are following the standard also followed by other Novartis projects including Info1, but only the parameterized values may vary from project to project.

Technologies Used:

Ascential DataStage Server Edition
Operating System - UNIX
User Interface – Microsoft Excel Template
UNIX Shell scripts
Execution Steps:

Enter the parameter details into the Excel template worksheet under the respective column header (if a particular parameter is not required then leave it empty)
Now save this excel file as a .csv file (comma delimited / separated values file)
FTP this file to the UNIX home directory for the tool(This path is also given at Datastage job level)
Check the files HEADER, FOOTER, UTILITIES etc to see if they contain the correct text. (These are UNIX files which will be used to generate the content for script)
Now run the job sequence to generate the whole script or execute the main job to generate a particular sub – step
The generated file is available at the path specified (The path can be defined in the Datastage Job for file generation)
Advantages:

Flexible Options – This tool enables the user either to generate an entire Control M Shell script / Simple shell script or simply a single Sub – Step for the same
Simple Interface – A simple Microsoft Excel template accepts the parameters for a step
Modular design – The Entire design is divided among different DataStage jobs to try and make it less complex and more maintainable
Support for best practice – This tool has been mainly designed to ensure that any script generated using the tool contains minimum “hard coding” or logical errors
Minimization of coding errors – Since the tool uses standard templates, once the templates are assured to be bug free the chances of propagation of bugs into the script are reduced
Balance between Time and Space / Resource complexity – This tool helps works on unix string formats only so the time required for generating is script is at most a minute or more and it does not require many resources also
Reduce coding time and efforts – Once the parameters are supplied to the tool, using a single job sequence all the activities are completed hence the majority of time required can be focused on setting the input parameters appropriately, as compared to the regular process of script coding this tool will require only ¼th of the time
Reusability – This is the key to generation of the tool, with a correct understanding of the structure of the Control M Script, this tool can be used across the practice to generate scripts as per their requirements
Future Scope:

Currently the tool can generate scripts for DataStage jobs only, but with a little modification and improvements it can be used to generate File watcher scripts and Archiving scripts too
Currently the tool can read data only from UNIX file sources, this can be further improved to read data directly from windows OS
Design of a better web based interface to improve the quality of the interface and make it further readable and simple
Cost Benefits:

This process of creating Unix shell scripts, used to run the ETL jobs has helped reduce the development efforts of the team by 50%.