How to modify the default e-mail behaviour of Cognos 8.3 and Cognos 8.4

Many jobs/reports stay in pending and SMTP queue status in Cognos Administration => the System shows a high number of queued emails.

Default email behaviour in Cognos 8.3 and Cognos 8.4 is to retry to send an email in case of a problem like wrong email recipient or attachment size limit exceeded etc. This causes problems in the SMTP queue (blocked) and scheduled jobs/reports stay in status pending, blocking other reports to finish successfully.

Default behaviour of Cognos 8.3 and Cognos 8.4
Check the SMTP queue as well as the number of pending jobs.

To add additional features and provide some more fine grain control to Cognos' e-mail functions, an advanced SMTP behaviour properties file has been introduced. The ability to modify the behaviour of the delivery service to a specific mail server exception code is controlled by a properties file in addition to the ability to change failure message recipients, failure message subject, invalid recipients, and retry attempts.
To begin to use a SMTP properties file, you will need to create two advanced delivery service properties and create your custom rules file.

Copy the file (smtpRules-default.xml.sample) attached to this Kb into install\webapps\p2pd\WEB-INF\classes directory and rename to smtpRules-custom.xml
1. Start Cognos Connection.
2. In the upper-right corner, click Launch, Cognos Administration.
3. On the Status tab, click System.
4. From the All Servers drop-down menu, click Services, DeliveryService.
5. From the drop-down menu for DeliveryService, click Set properties.
6. Click the Settings tab.
7. For the Environment category, next to Advanced settings, click the Edit link.
8. If it appears, select the Override the settings acquired from the parent entry check box. Otherwise, proceed to the next step.
9. Enter parameters and values shown below, changing the path as required.
10. Click OK.

(Remember, do not use quotes in the *.xml files) - /smtpRules-custom.xml - true

NOTE: Use "true" only for test purposes because this causes the DeliveryService to check the file each time it has to send an email. This impacts performance. When the modification of the smtpRules-custom.xml is finished please set '' to "false" and restart DeliveryService.

The '' simply targets your custom rules file and the '' fadvanced property set to true forces the rules file to be reread for every request.(allows changes to be made without a restart) It is advised to change the reread property to false after you are happy with the functionality as this reread will cause a small performance hit.
With the advanced properties now set, you will need to create/modify the rules file to address the customization or functionality change specific to your requirement. Functionality changes are controlled by the '/smtpRules-custom.xml' file which you targeted in the steps above. In this default smtpRules-custom.xml file itself and below you will find examples and explanations/definitions of the various tags and their usage.
NOTE: The priority of rules is determined by the order in which they appear in the XML file.

You can define the following types of SMTP error:

transport errors
For example, no connection to the mail server, the mail server does not exist or is not configured correctly, or the user has no access to the mail server. Use
to include this type of error in your rules.

recipient errors
For example, invalid recipients, too many recipients, or no recipients. Use
to include this type of error in your rules.

other specified errors
Any standard SMTP error code generated by the mail server. Use
to include this type of error in your rules. The following actions can be performed for each error type, and are defined as behaviours in the XML file

resend behaviour
Specifies how many times to resend an email (n) and the resend interval in seconds.
NOTE: this "resend" and "delay" does not work in conjunction with 8.3 RTM, 8.3 SP1 and 8.3 SP2, Do not use it to avoid an infinite loop. It works fine and can be used in 8.3 SP3, 8.4 GA as well as 8.4 FP1.

default smtp behaviour
This definition should be placed at the end of the file to process an error code that does not match to any previous smtpRule. Use:
to include this type of error in your rules

keep mail behaviour
Specifies whether the delivery service should keep the failed email in a separate queue after it has been resent the required number of times, and is unsuccessful. The queue is namedSMTPBackupQueue.
NOTE: No further action is performed on emails in the backup queue. To add emails from SMTPBackupQueue to the regular SMTPQueue, you must change the queue name in thedatabase table and restart the server.Use <keepMail>true</keepMail>
to apply this behaviour.
Recommended setting: false

fail mail behavior
Allows you to customize the email notification that is sent when an email delivery has failed. Use the:
<failMail> tag to apply this behavior. There are two further optional attributes you can use to specify the email notificationsubject which is <subject> and <recipient address>
Tip: If you omit these tags, by default, the email notification is sent to original recipients list with the subject "Send failed:".To remove all current recipients, use
<recipients sendToCurrentRecipients="false"
To send an email notification to the agent owner, use
<owner>true</owner> and, if required, use <recipient address="name@address> to specify an email address.
NOTE: Today, please do not use this <owner>true</owner> in any smtpRule due to a bug in this function.

Cognos 8.3: does not include this smtpRules-default.htm by default. This file is attached. A further file with name smtpRules.xsd is need which is attached as well. Both need to be copied into the install\webapps\p2pd\WEB-INF\classes
Functional restrictions: In 8.3 RTM to 8.3 SP2 do not use resend behaviour to avoid an infinite loop. Do not use <owner>true</owner> at all.

Cognos 8.4: includes this file by default. Administration and Security Guide includes documentation about it. No smtpRules.xsd is needed. Do not use <owner>true</owner> at all.