Use SIL to the maximum

Hello!

In this article I would like to talk abut SIL. We all know SIL because of its routines which let us automate our actions in Atlassian Jira and Confluence. We write our routines in the SIL Manager menu option and most of us do not use other menu options which are available for SIL. Here are all these menu options:

In this article I will discuss all menu options except SIL Manager to let you know how many more features are available in SIL.

All these options are available for the SIL engine which means that it is all free for you.

Custom Field Usage

This menu option provides you with information on how each custom field is used in your Jira. Vendor’s information is available here.

You can find all custom fields in the cog item -> Issues -> Custom Fields menu.

Here is how the screen looks if you click the Custom Field Usage menu option:

As you can see you have a select field called Custom Field which contains the list of all custom fields in your Jira. The list shows a limited amount of custom fields that is why you can start typing the name of the custom field you need and the list will be filtered by the letters you typed:

After you selected the field you need to click on it:

You can see the SIL aliases for the field, screens and projects where the field is used and also the lines of code where the custom field is used.

At the moment of writing this article there is an open bug CADS-6237 which says that the References part is not showing lines of code where the custom field is used. That is true. I had to modify the code of the SIL engine to make the References part work properly for this article that is why in the screenshot above the References part contains rows with scripts where the custom field is used.

Redirect Page Configuration

This menu option lets you make a smart redirection to Jira pages. You can find Vendor’s information here. To tell you the truth, I spent a couple of hours to understand why we need this feature and how to use it. I feel that it is better to explain this feature with an example. Here is the example.

Suppose, each employee files invoices each month in Jira.

As you can see Alexey Matveev filed two invoices for March and April.

But HR works in an external system where the payment is done and in this external system HR officer must be able to see the latest invoice, which was filed in Jira, for each employee.

Of course this HR officer can login to Jira, open Issue Navigator and filter all issues in the Invoices project by employee ordered by the Invoice Date custom field in the descending order and the first invoice will be the latest invoice.

project = INV and reporter = amatveev order by "Invoice Date" desc

But it is a too complicated approach. We would like to be able to access the latest invoice by a link in our HR system.

But what will be the link?

We can not make this link look like this: http://localhost:2990/jira/browse/INV-2. Because this invoice will be the latest in May but not the latest in June. Do we need to change this link every month in our HR system? It sounds too complicated! We need a simpler solution.

And the Redirect Page Configuration will help you here.

Let’s configure it.

In our HR system we can make the link look like this:

http://localhost:2990/jira/plugins/servlet/kredi?userName=Alexey+Matveev

As you can see we call the plugins/servlet/kredi servlet which will do smart redirection for us and also we pass Alexey+Matveev as the user name parameter which will tell the servlet for which user we need the latest invoice.

Now we need to make our servlet smart. Open the Redirect Page Configuration menu and enter the following code:

string userName = argv["userName"];

if (isNull(userName)) {
    return "/plugins/servlet/kredierror?customErrorTitle=User Not Provided&customErrorMessage=Provide user with userName parameter";
}
if (isNull(getUserByFullName(userName))) {
    return "/plugins/servlet/kredierror?customErrorTitle=User Not Found&customErrorMessage=User " + userName + " not found";
}
string [] k = selectIssues("project = INV and reporter = " + getUserByFullName(userName).username + " order by \"Invoice Date\" desc");
if (size(k) == 0) {
   return "/plugins/servlet/kredierror?customErrorTitle=Invoice Not Found&customErrorMessage=No invoices for "+ userName; 
}
return "/browse/" + k[0];

The script does the following:

we take the value of the userName parameter which was passed in the url.
we check if the userName parameter was really passed. If it was not passed we redirect to the error page with the message “Provide user with userName parameter”.
we check that a user with the user name exists in Jira. If not, we redirect to the error page with the message “User userName not found”.
we select the latest invoice for the user. If no invoices selected, we redirect to the error page with the message “No invoices for userName”.
we show the invoice page.

Well, let’s test it!

First let’s make a link like this:

http://localhost:2990/jira/plugins/servlet/kredi?userName=Alexey+Matveev

If we click on this link we will see the following screen:

Correct! We were redirected to my latest invoice.

Now let’s not pass the userName parameter:

http://localhost:2990/jira/plugins/servlet/kredi

And we were redirected to the error page:

Correct!

Now, let’s provide the user name who does not exist in Jira

http://localhost:2990/jira/plugins/servlet/kredi?userName=Super+Man

We were redirected to the error page:

And now let’s provide a user without filed invoices:

http://localhost:2990/jira/plugins/servlet/kredi?userName=Tomas+Brook

Again we were redirected to the error page:

So, our program works correctly!

In short, we moved all logic for redirecting us to the correct page to Jira. And it is very handy because we can implement our logic with the SIL language which makes manipulation with Jira easy.

Switch User

This menu let us work in Jira under any registered user in Jira without knowing the password of the user. You can find Vendor’s information here.

The list of users is limited but you can type letters in the User field and users will be filtered for you:

Click on the user you want to work under, push the Switch! button and you will be logged in as this user without providing the password of this user.

To return back to your user you can use the Switch Back To option:

SIL Configuration

This menu option lets you set configuration parameters for SIL. You can find Vendor’s information here.

SIL Home Directory lets you change the file folder where all sil scripts will be placed. By default all scripts are placed to your JIRA_HOME/silprograms folder, but you can place the scripts into any folder you want under the JIRA_HOME folder.

Charset lets you define the character encoding for your scripts. I think it is better to leave it as it is UTF-8. Maybe you would need to change it if you save your sil scripts in the Jira database and somehow your database character set is not UTF-8. But I would say it is a very rare case. You should use UTF-8 in all your Jira settings.

SIL Cache size lets you define the number of the parsed SIL scripts in the SIL cache.

Why would we need a SIL cache at all?

Here is a simple script in sil:

string message = "My message";
runnerLog(message);

In order to execute this script SIL must convert this script to Java byte code. There are many different constructions in SIL like structures, functions, inclusions, loops, conditions, assignments, variable type conversions and so on. All of it must be converted from SIL code to Java byte code. And it all takes time. That is why once SIL code is parsed the Java byte code is saved in the SIL Cache to save your time on parsing this SIL code again.

That is why if you have thousands of scripts it makes sense to try to increase this parameter and have a look if you have performance gain.

Datasources

This menu option lets you add database source which can be used later in your scripts. It is very handy because you do not need to hardcode database connection and user password in the code. You can find Vendor’s information here.

Let’s configure a database source and use it later in sil scripts.

Push the Add Datasource button and connect to an external database:

I put up a second Jira instance in my notebook and connected to the database of the second Jira instance.

It is important to mention that SIL does not contain jdbc drivers in its distribution that is why you need to make sure yourself that you have an appropriate driver in your Jira instance. You can read here about adding jdbc drivers to Jira.

Now let’s push the Save button and we will have a new data source called external_database configured:

Now let’s use it in our SIL script:

string [] results = sql("external_database", "select * from cwd_group");
runnerLog(results);

I executed an sql query on my external database and received a result:

Mail Sender Configuration

It is a tricky one. You can find Vendor’s documentation here.

You can send a email with the sendEmail SIL routine and this menu option lets you provide parameters for this routine.

My templates directory provides the path to the email templates.

Mail language on lets you define the language of the sent email by Sender or Receiver. If you use email templates you can create templates for multiple languages. You can read more here.

Send mail via lets you define how your email will be sent. Here are the available options:

Container Sender – sends a message by adding your email into the Jira mail queue.

And you can see error and debug messages in the atlassian-jira-outgoing-mail.log if you have properly configured your logs. You can read more on how to configure email logging here. I prefer to use this option.

Direct sender, custom – you can define your own SMTP mail server:

You will not see the email message in the Jira mail queue or logs in the atlassian-jira-outgoing-mail.log file, but you will be able to see log messages in the atlassian-jira.log file.

Direct sender, default – you can use the SMTP server defined in Jira. The email message will be send by SIL. You will not see the email message in the Jira mail queue or logs in the atlassian-jira-outgoing-mail.log file, but you will be able to see log messages in the atlassian-jira.log file.
Null sender (log only) – the message will be written in the logs (it will not be sent to the receiver). But surprise, surprise! The level of the log is INFO, but the default log level for the com.keplerinfo package is ERROR. And as a result no logs in the log file, no emails in the email box and this option is the default one. I spent an hour to figure out how to make everything work. Just add the INFO log level for the com.keplerinfo (more info here) or choose another option. Here is an example of a log message:

2020-05-24 14:19:53,601+0300 pool-42-thread-8 INFO admin 859x6032x1 aizaws 0:0:0:0:0:0:0:1 /rest/keplerrominfo/refapp/latest/async-script/runScriptFromEditor [c.k.r.sil.impl.MailConfigurationAccessor] NULL MAILER (log only mail sender) : Subject: aa, From: null, To: [alex@gmail.com], CC: [alex@bk.ru], Body:
aa

Asynchronous Runner

If you click on this menu option you will see the following screen. You can find Vendor’s information here:

Threads sets the number of threads for SIL engine. By default it is set to 10 which means that you can not execute more than 10 SIL scripts concurrently. If you have many scripts running at the same time consider to increase this value.

Time To Live (TTL) sets the time for a thread to live. The default value is 1 hour. It means that if a script runs more than 1 hour it will be killed. If you have long running scripts consider to increase this value.

Checkpoint Interval sets the time how often SIL engine will check running threads and kill the threads if the threads have been running for more than TTL time.

Also you can see all running tasks in the Running tasks part of the screen.

Remote Systems

The Remote Systems menu option lets you

Add connections to remote systems and call sil scripts from these remote systems.
Set permissions to call SIL scripts by SIL REST API.

You can find Vendor’s information here.

Let’s first create a connection to a remote system and call a sil script from this remote system.

I have two Jira instances running on localhost:2990/jira and localhost:8080.

I have this SIL script on localhost:8080 called test.sil:

logPrint("ERROR", "I am called from " + argv[0]);

This scripts accepts a parameter and logs an error message into the atlassian-jira.log file with a message which includes this parameter.

Now on localhost:2990/jira I go to cog item -> Manage Apps -> Remote Systems, push the Add Remote button and provide data for my localhost:8080 Jira instance:

Now push the Save button and create a script on localhost:2990/jira which will call the test.sil from localhost:8080:

call("my_ext_jira", "test.sil", "localhost:2990/jira")

The first parameter is the name of our connection (my_ext_jira).

The second parameter is the name of the script which will be executed on the remote system.

The third parameter is the additional parameter which will be passed to our test.sil and we will take the value of this parameter as argv[0].

Now let’s run the script and have a look in the logs on localhost:8080. We will see the following line:

2020-05-25 08:30:57,944+0000 pool-38-thread-2 ERROR admin 510x101x1 3sauem 172.26.0.1 /rest/keplerrominfo/refapp/latest/async-script/runScript [c.k.s.lang.routines.LogPrintRoutine] I am called from localhost:2990/jira

It means that we successfully called a SIL script on localhost:8080 from localhost:2990/jira.

Next let’s have a look at the Security section of the Remote Systems menu option.

Suppose we have two scripts on our localhost:2990/jira.

test.sil

call("my_ext_jira", "test.sil", "localhost:2990/jira")

test1.sil

runerLog("Hello World");

And also I have a user called user1 who does not have the Jira Administrator permission and I want this user to be able to execute inline scripts and sil scripts on the file system by SIL REST API.

Let’s go to Remote Systems:

We will be working with the Grant execute inline, Grand read and Grant execute buttons from the Security section.

First let’s try to execute an inline script by user1.

http://localhost:2990/jira/rest/keplerrominfo/refapp/1.0/async-script/runScript

With this JSON:

{
   "source" : {
    "type": "INLINE",
    "code": "return 1;"
    }
}

And we have the 403 Forbidden error.

Now let’s push the Grant execute inline button and add this grant to user1:

And now we got the 200 response which means that the script was executed successfully.

Now let’s try to execute the test1.sil file under user1. This time the JSON will look like this:

{
   "source": {
        "type": "FILE",
        "code": "test1.sil"
    }
}

And we get the 403 Forbidden error. Now let’s permit user1 to execute the test1.sil file, push the Grant execute button:

Let’s execute the script and this time the response code is 200.

LDAP Configuration

You can provide a connection to your LDAP. You can find Vendor’s information here.

I run a docker with a ldap server from here.

docker run -p 389:389 -p 636:636 --name my-openldap-container --env LDAP_ADMIN_PASSWORD="adminadmin" --detach osixia/openldap:1.3.0

Let’s push the Add LDAP button and configure our connection:

The only available option for the Directory field is Active Directory. But it does not mean that you can make a connection only to Microsoft Active Directory. You can use any LDAP server. In my case I used open ldap.

Now we can select data from this ldap.

Let’s try to select data with the ldapUserList routine:

runnerLog(ldapUserList({"cn", "uid"}, "objectClass=*", "myldap"));

And I successfully chose the admin user from my ldap.

Script Storage

This menu option lets you define where to store your sil scripts. You can find Vendor’s information here.

By default all SIL scripts are stored in the file system (the disk option):

If you choose the database option all scripts will be stored in the AO_1B54DA_TSTEXT database table.

I believe that it is better to save your SIL scripts on the filesystem because in this case you can store all your scripts in a version control system. For example, Bitbucket.

Custom Fields Mapping

You can map values of custom fields in your Jira to SIL values. You can find Vendor’s information here.

For example, let’s have a look at the Checkboxes custom field. The value is mapped to the string[] SIL value which means that you can get a value from a checkbox field like this:

string[] value =  #{My Checkbox Field};

And you can set a value to a checkbox field like this:

string[] value =  {"value1", "value2"};
#{My Checkbox Field} =  value;

You can not change the mapping for the field because it is a standard Jira field and Cprime already provided the correct mapping for you.

Now please, pay attention to the Grid custom field (the second row in the red rectangle). This grid custom field goes from the Table Grid Next Generation add-on. Cprime does not provide a mapping for this field by default. That’s why it is mapped to the string value by default but you can change the mapping:

In some cases any available out of the box mapping will not work correctly. In this case you can develop a custom plugin for SIL with your own mapping. You can find more information here.

Intergration Configuration

You can create connections to Slack and Stride here and then use these connections in your SIL scripts. You can find a very good explanation how it works here.

SIL Webhooks Configuration

You can create your own REST API in this menu option. You can find a detailed explanation here.

SIL Diagnostic

This menu provides you with the overall information about your SIL engine:

I believe that most of this information is needed for developers of the SIL engine, but I would like to mention a very cool feature here.

You can see all SIL threads and kill them.

Suppose, you wrote and ran a SIL script which keeps executing and takes lot’s of cpu or memory resources. You can kill this thread from here.

You can see that I have a thread with TID 393 and I know that this script just takes resources, I do not need this thread. All I need to do is to push the kill button. And the thread will be killed.