Introduction:-

Validation transform is used to filter or replace the source dataset based on criteria or validation rules to produce desired output dataset.

It enables to create validation rules on the input dataset, and generate the output based on whether they have passed or failed the validation
condition.

In this Scenario we are validating the data from the database table with correct format of the zip code.

If the zip code is less than 5 digit then we will filter that data & pass it to another table.

The Validation transform can generate three output dataset Pass, Fail, and RuleViolation.

1. The Pass Output schema is identical with the Input schema.
2. The Fail Output schema has 3 more columns, DI_ERRORACTION and DI_ERRORCOLUMNS, DI_ROWID.
3. The RuleViolation has three columns DI_ROWID, DI_RULENAME and DI_COLUMNNAME.

Steps:-

1) Create project, job, workflow, dataflow as usual.

2) Drag source table, Validate transform& provide details.

• Double click on Validation transform to provide details. You can see the 3 types of dataset as described above.

• Add a validation rule.

• Click Add & fill the details about the rule as follows.

Action on Fail:-

                1) Send to Fail:-  on failure of the rule the record will sent to another target with "Fail" records.

                2) Send to Pass:- even on failure pass the record to the normal target

                3) Send to Both:- sends to both the targets.

Column Validation:-

                Select the column to be validated, then decide the condition.

                We have selected "Match Pattern"as the condition  pattern as '99999'.

                So it will check whether Zip code is of 5 digits or not.

• Press OK. Then you can see the entry get added as follows.

3) Add a Target table to the dataflow & link the Validate Transform to it.

• Choose the validate condition as "Pass"

• Similarly do the connection for "Fail" & "Rule Violation" condition.

4) Validate the job & execute it.

5) Check the input & output.

• Input:-

• You can see in the input in the above figure where the last row has zip code of less than 5 digits. Now view the output.

• Output for Pass condition:-

• Output for Fail condition

     You can see that the invalid record from input is transferred to the  "CUST_Fail" table as shown above.

     Three more columns "DI_ERRORACTION", "DI_ERRORCOLUMNS", "DI_ROWID" can also be seen.

• Output of the "RuleViolation" condition.

Summary:-

So in this way Validate transform is useful in validating the records based on the rules & categorising the bad records into different target which can be analysed later.

Thanks & Regards,

Rahul More

(Project Lead)

Hi Team,

It's a basic requirement but most common .So I wanted to share it for beginners.

Purpose : Job should execute for a selected country .If there are no records for the selected country then job should terminate .

Create 2 global variables to at Job level

$CNTR : Country to select when job execution

$RC : count records for selected country.

1.Scr_Records : to count the records for a selected country.

$RC = sql('SRT','SELECT Count(*) FROM Stadium_Info where COUNTRY={$CNTR}');

Print('Rows'||$RC);

2.Add try and Catch to display the error message in Trace log

3.Add Conditional (If) transformation .

        In Condition part , write $RC >0

        * Then part : Create your dataflow when condition met .Put Stadium_Info.COUNTRY =$CNTR in transformation WHERE clause to restrict rows for selected country.

        *Else part  : Create a script to generate error

         Scr_Raise_Exception :  raise_exception( 'Table is Empty');

4. In Catch block :

Add a script  to Print your error message in Error log

raise_exception( 'There arer no records to fetch for '||$CNTR);

5.Execute the job and enter value for Country variable $CNTR ='IN' ( in database there are no records for IN)

observe Trace and Error logs.

Hope it helps somebody need.

The error "The product key is not valid. INS00140" is sometimes produced on Unix/Linux platforms during the installation of BO products and  it  is most likely to be due to a missing library/libraries.

First you need to make sure that the used key is actually valid, then follow these troubleshooting steps to find out what exact libraries or their dependencies that  are missing:

(NB: if in doubt about the license key validity, use Temporary License Key, this is perpahs  more relevent  to BI platform, as IPS uses an embeded key but the latter can also be tested with the temp key).

1. cd "/<Install_media_location> /IPS/DATA_UNITS/IPS_lnx/dunit/product.minibip-4.0-core-32/actions/"  This path is   for IPS 4.1, but BI,  DS and IS should have   other path for their own license program.
2. export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH. This sets the environment variable for the libraries required for the key validation and ‘pwd’ is meant for the path in step 1 above.
3. ldd ./isKeycodeValid. This command will list all the libraries required for this execution.
4. Run ./isKeycodeValid"  -keycode XXXX-XXXXXXX-XXXXX-XXXXXXX-XX -version 140 -property BOE.EnableBO. This validates any key for IPS and you would expect to  an error poiting to a missing library.
5. When getting an error in step 4 make a note of the library.
6. Run commad ldd <missing_library_name> to print the shared library dependencies
7. If you know that the libraries  and  their  dependencies, actually exist in a specific location, then you need to  update the $LD_LIBRARY_PATH with their path before you can perform the install.
8. Once you identify the missing libraries from the system ie. libstdc++XXXX, try to find out the package name in Linux which includes them (ask system admin or Google for the rpm name) and install it. It would most likely be a missing 32 bit libraries in a 64bit system.

If a similar error is encountered in DS:

1. Navigate to the LicenseManager path
2. ldd <Install_media_location>/dunit/product.dataservices64-4.0-core-32/actions/utilities/LicenseManager
3. The result from the above step may look like this:

     4.  You can notice that libboeim_kd_64.so is  not found

     5.  On this occasion this library is found on the installation media when searched for,  so only the $LD_LIBRARY_PATH needs to be added/appended.

     6.  export LD_LIBRARY_PATH <Install_media_location>/dunit/product.dataservices64-4.0-core-32/actions/utilities/

     7. The LicenseManager/Installation would now be able to validate the key without errors.

Your comments are welcomed.

Nawfal Tazi

Purpose:

The purpose of this document is to up-grade SAP Business Objects Data Services from 11.7/3.X/4.X to SAP Data Services 4.2 SP05

Overview:

Environment Details:

Operating system: Windows Server 2008 64 Bit

Database:Microsoft SQL Server 2012 R2

Web Application:Tomcat

SAP Business Objects Tools:SAP Business Objects Information Platform Services 4.1 SP02; SAP Data Services 4.2 SP05

Migration Tools:Data Services Repository Manager

Repository version:BODI 11.7 or BODS 3.X or BODS 4.X

Installation & Configuration steps to upgrade SAP BODS 3.X/4.X to SAP BODS 4.2 SP05:

Pre Installation Check List:

• Backup of repositories, configuration files & Data Cleanse files:
• Create a Check list of Number of Batch Jobs/Real Time Jobs, Number of Access Server, Number of Job Server & Configuration Path (Datastore) of Source & Target Systems
• Create a Check list of Jobs Scheduling in SAP Business Objects Data Services Management Console
• Create a Check list of Groups & Users Available in SAP Business Objects Data Services Management Console

• Backup of Local, Profile & Central Repository
• Backup of Following Configuration Files:
  • admin.xml
  • sapconnections.xml
  • as.xml
  • dsconfig.txt
• Backup of Data Cleanse Files

Post Installation Check List:

• Installation of EIM(Enterprise Information Management) or IPS(Information Platform Services) Package in New Landscape
• Installation of SAP BODS 4.2 SP05
• Installation & Configuration of Address Cleansing Package 4.2
• Best Practices says use the same Data base that you have use in SAP BODS 4.x make a cloned copy and Use in SAP BODS 4.2 SP05 environment
• Up-gradation of Local /Central repository Using Data Services Repository Manager
• Configuration of Repository in EIM or IPS (Package Name) Central management Console.
• Configuration of SAP RFC in Data Services Management Console 4.2 SP05
• Configuration of Adapters in Data Services Management Console 4.2 SP05
• Apply Rights & Security according to your requirement in BIM or IPS (Package Name) Central management Console
• Now Login in SAP BODS Designer 4.2 SP05 and configure Jobs (Real Time/Batch Jobs)
• Configuration of Datastore (SAPR3,SAP BW, RDBMS), Substitution Parameters, Job Server, Access Server, Format Data Type (Flat Files, CSV)
• Now validate the Jobs at Top Level and if error comes fix those errors
• Now Execute the Jobs using Scheduler or other BI Tools

SAP Business Objects Information Platform Services 4.x SPxx Installation & Configuration Documents:

Product Infrastructure Dependency:

Data Services (DS) requires SAP BusinessObjects Business Intelligence (BI) or Information Platform Services (IPS) for security capability. The table below shows the BI/IPS version and the corresponding compatible version of Data Services.

For more information on the compatibility of Data Services with BI platform or IPS versions, and compatible patch information, please see SAP Note

1740516 - SAP Data Services 4.x and SAP Information Steward 4.x compatibility with SAP Business Intelligence platform and Information Platform Services for active releases

https://service.sap.com/sap/support/notes/1740516

1842698 - SAP Data Services 4.2 Upgrade Scenarios

https://service.sap.com/sap/support/notes/1842698

Supported Platform Link:

http://service.sap.com/PAM

Validation of Current Environment before and after application upgrade:

For Validation of Jobs, Access Server, Adapter & Web Services I am using script to populate the data in text file, Please refer the o/p for your reference.

Count of Repository: Currently I have 1 “Local & 1 “Secure Central repository” highlighted in Yellow color

* For validation you can also use “Local repository” metadata for above details. Here I used BATCH script to check all above parameter in my current systems, Please refer the below screenshot for your reference.

Count of Job Server:

Count of Access Server:

Count of Batch Job:

Count of Real Time Job:

*Same details you can get from batch script for Adapter & Web Services.

Up gradation & Configuration Documents of Data Services 4.2 SP:

When the prerequisite system check completes successfully, click Next

On the "SAP BusinessObjects Data Services 4.2 SP05 setup" page, click Next.

Accept the License Agreement, and click Next.

Click Next to accept the default path to the folder where the program will be installed.

Click Next to accept the default language (English).

Type a password for the CMS Administrator account, and click Next.

Click Yes to restart the BOIPS services

To start the installation, click Next.

Installation is in progress

Finish

SAP Business Objects Data Services 4.2 SP5 Configuration Documents:

Before upgrade of local repository below are the error in SAP Business Objects Data Services Designer

So before login upgrade the repository using SAP Business Objects Repository manager below are the steps for upgrade of  local repository details:

Below are the steps for upgrade of local repository & upgrade of secure central repository

Upgrade of Local Repository using SAP Data Services Repository manager:

Open SAP Data Services repository manager and check the version before migrating, You can also check in repository metadata through SQL

And after validating click on upgrade tab

Upgrade of Secure Central Repository:

Do the same validation for Secure central repository

And after validating click on upgrade tab

Then Login to Central Management Console -> Go to Data Services

Then Click on Data Services and check the repository version details.

Now try to login in SAP Business Objects Data Services Designer with SAP Business Objects Credential

Click on Logonand Click on “ok”

Now login in SAP Business Objects Data Services management console

Click on "Administrator" tab

New features are added in “SAP Business Objects Data Services management console 4.2":

• Object Promotion:
  • Import Configuration
  • Export Configuration
• Export Configuration using two ways:
  • FTP
  • Shared Directory
• Substitution Parameter: Now we can change the "Substitution Parameter" settings through SAP Business Objects Data Services Management console also.
• New Features added in Adapter like "Hive Adapter & VCF Adapter"
• Changes in "Query and Validation" transform in SAP Business Objects Data Services Designer 4.2
• New Features added like "REST web services": Representational State Transfer (REST or RESTful) web service is a design pattern for the World Wide Web. Data Services now allows you to call the REST server and then browse through and use the data the server returns
• Changes in "Architecture" of SAP Business Objects Data Services please refer below upgrade guide for more details.

After Installation & Up-gradation SAP Business Objects Data Services 4.2SP05 below are the status that i upgraded in VM Ware

Then again upgrade from 3.x to 4.x

Relevant Systems

These enhancements were successfully implemented in the following systems:

• SAP Business Objects Information Platform Services 4.1 SP02/SAP Business Objects Enterprise Information Management 4.1 SP02
• SAP Business Objects Data Services 4.1 SP05

This document is relevant for:

• SAP Business Objects Data Services Administrator

This blog does not cover:

• SAP Business Objects Data Quality Management Up-Gradation

Reference Material:

Upgrade Guide:

https://help.sap.com/businessobject/product_guides/sbods42/en/ds_42_upgrade_en.pdf

Installation Guide Windows:

https://help.sap.com/businessobject/product_guides/sbods42/en/ds_42_install_win_en.pdf

Installation Guide UNIX:

https://help.sap.com/businessobject/product_guides/sbods42/en/ds_42_install_unix_en.pdf

SAP Data Services 4.2:

https://help.sap.com/bods

SAP Data Services 4.2 Product Availability Matrix:

https://websmp208.sap-ag.de/~sapidb/011000358700000441762013E

SAP Data Services Blueprints (Sample ATL Files of BODS 4.x)

Links

http://scn.sap.com/docs/DOC-8820

Data Services Product Tutorials:

Links

http://wiki.scn.sap.com/wiki/display/EIM/Data+Services+Product+Tutorials
http://wiki.scn.sap.com/wiki/display/EIM/Data+Services

Video Tutorials:

Links

http://wiki.scn.sap.com/wiki/display/EIM/Video+Tutorials

SNotes:

1388363 - Where can I download Data Services software, support packages, and patches?

https://service.sap.com/sap/support/notes/1388363

1263925 - Are Data Quality blueprints available? - Data Services

https://service.sap.com/sap/support/notes/1263925

1530081 - Troubleshooting tips for SAP Data Services 4.x

https://service.sap.com/sap/support/notes/1530081

1723342 - Data Services 4.1 Upgrade Scenarios

https://service.sap.com/sap/support/notes/1723342

1725915 - Release Notes for SAP Data Services 4.1 (14.1.0) Ramp-Up

https://service.sap.com/sap/support/notes/1725915

1740516 - SAP Data Services 4.x and SAP Information Steward 4.x compatibility with SAP Business Intelligence platform and Information Version

https://service.sap.com/sap/support/notes/1740516

1900051 - SAP Data Services 4.x and SAP Information Steward 4.x compatibility with SAP BusinessObjects Business Intelligence platform and Information platform services for inactive releases.

https://service.sap.com/sap/support/notes/1900051

1694041 - BI 4.0 Consulting:- How to size the Adaptive Processing Server

https://service.sap.com/sap/support/notes/1694041

1881527 - BODS adapter issues

https://service.sap.com/sap/support/notes/1881527

1769406 - Best Practices on taking a Hot Backup for a SAP BI 4.0 deployment

https://service.sap.com/sap/support/notes/1769406

746938 - How to backup or restore a Data Services 4.0 cleansing package.

https://service.sap.com/sap/support/notes/746938

DS4.1 - Address Cleansing using Global Address Cleanse transform

http://scn.sap.com/thread/3316670

1645932 - How to setup SSL certificates in a Distributed Environment - Data Services XI 4.0

https://service.sap.com/sap/support/notes/1645932

1416303 - Can't reach Job Server from DS Designer

https://service.sap.com/sap/support/notes/1416303

Address Cleanse Best Practices for India

Input Format

The global address cleanse transform deployed within Data Services or DQM SDK supports a wide variety of input fields that can be mapped to an incoming data source.  The input fields can be categorized as multiline, discrete or hybrid.

Multiline fields can generally be used when the incoming data is in a mixed format and is not properly fielded within the source itself.  Typically this format of data isn’t widely used.  The global address cleanse transform names these fields as “MULTILINE[1-12]”.

Discrete fields within the global address cleanse transform represent data in an input source that is properly fielded.  This is the most typical format of data used by our customers.  For this use case, the input source will have address data in columns that generally describe the data component such as locality, region, country and address line.  These fields are named “LOCALITY[1-3]”, “REGION”, “POSTCODE”, “COUNTRY”, “ADDRESS_LINE”.

The hybrid format uses both multiline and discrete fields to give greater flexibility to map the data from the input source to the global address cleanse transform.

The standardized and cleanse results may vary slightly based upon which input format is used to map the data into the global address cleanse transform.  The reason for this is that when the input data doesn’t have an exact format (MULTILINE), we need to do additional parsing to determine what the various address components are.  The discrete method generally may be more accurate.

The goal of the different input formats that are supported is to give flexibility to the developers leveraging the global address cleanse transform.  The same can be said about the output fields that are supported, which will be covered below.

Output Fields

The global address cleanse transform supports both discrete and compound output fields.  Discrete output fields represent a singular entity with an address.  An example of this would be the PRIMARY_NAME1 (street name) output field.  Compound output fields represent a group of discrete fields.  An example of this would be the PRIMARY_NAME_FULL1 output fields.  This output field has the following structure:

PRIMARY_PREFIX1 PRIMARY_NAME1 PRIMARY_TYPE1 PRIMARY_POSTFIX1

The global address cleanse transform is able to standardize and cleanse data on a country-by-country basis using purchased postal address files.  The data that is output to the fields supported depends on 2 factors:

1. The address data that can be parsed and identified using the input data
2. The level of detail of the address data in the postal address file

Parsing and Identification

There are instances where the input data can be identified and parsed into discrete or compound components, but the data may not be standardized or corrected.  For instance, the global address cleanse transform may be able to identify that a building name was found, but the following may happen:

1. The building name could not be found in the postal address file due to the quality of the data being so low a match couldn’t be made.
2. The postal address file does not have building name data (this is the case for India).

Output Components Supported for India

As mentioned above, the data that can be standardized and cleansed is directly related to the level of data that can be found within a specific country’s postal address file. An analysis was completed using the India address postal file and the following output fields below should consistently be populated.  There are a few caveats to this:

1. The data cannot be of such low quality that individual address components cannot be identified or parsed.  If this happens, then they will not be used in the assignment process.
2. If the data can be parsed and identified, then we can use it in the assignment process. However, using our fuzzy matching and lookup algorithms, similar data needs to be present in the postal address file for the standardization and cleansing to happen.
   1. Parsing of data uses key words in the input data that are flagged as building name, POR (Point of Reference used in many IN addresses, see example below), or Firm words. If these words are not present in the input we have no way of knowing that the data is a building, POR or Firm.

Output Fields with Cleansed Data – ordered from largest area to smallest

• COUNTRY
• REGION1_FULL
• REGION2_FULL
• POSTCODE1
• POSTCODE_FULL, which contains:
  • POSTCODE_DESCRIPTION AND POSTCODE_1
• LOCALITY1_NAME
• LOCALITY2_NAME
• LOCALITY3_NAME
• PRIMARY_NAME1_FULL, which contains
  • PRIMARY_PREFIX1
  • PRIMARY_NAME1, PRIMARY_TYPE1
  • PRIMARY_POSTFIX1
  • If the address is a postal address (e.g. PO BOX) then the following would be used: POST_OFFICE_NAME,  PRIMARY_NAME1
• PRIMARY_NUMBER
• PRIMARY_NUMBER_DESCRIPTION

Output Fields with Parsed Data

As mentioned above, there are caveats as to when the output fields are populated using the data from the postal address file.  The following fields will be populated if the address component was parsed and identified, but the reference data does not contains the level of detail to correct it properly.

• FIRM
• PRIMARY_NAME2_FULL, which contains:
  • PRIMARY_NAME2
  • PRIMARY_TYPE2
  • PRIMARY_POSTFIX2
• SECONDARY_ADDRESS, which consists of:
  • UNIT_DESCRIPTION
  • UNIT_NUMBER
  • BUILDING_NAME1
  • BUILDING_NAME2
  • BLOCK_DESCRIPTION
  • BLOCK_NUMBER
  • AREA_NAME
  • FLOOR_ DESCRIPTION
  • FLOOR_NUMBER
  • WING_DESCRIPTION
  • WING_NUMBER
• POINT_OF_REFERENCE
• ADDRESS_LINE_REMAINDER[1-4]
• LASTLINE_REMAINDER[1-4]
• EXTRA[1-12]

Address Line Remainder, Last Line Remainder and Extra data

An explanation has been given to describe how the data is populated and why based upon identification of address components and the level of detail found within the postal address file. This section will build on those concepts to give further insight into the address cleansing process to explain further where the input data goes if data could not be parsed or identified.

The ADDRESS_LINE_REMAINDER[1-4] output fields should typically be used in the following situation: address data that was not able to be identified that was found on the same input field as data that was parsed and identified will be output to these fields. This is done so that the address data on input is preserved.  The global address cleanse transform will not delete data that couldn’t be leveraged during the assignment process.  These fields should be referenced to further enhance your output if needed.

The LASTLINE_REMAINDER[1-4] function just as the ADDRESS_LINE_REMAINDER[1-4] output fields, but lastline data that was not able to be identified will be output within these fields so that all input data is preserved that cannot be corrected and/or standardized.

The EXTRA[1-12] output fields will be populated if the entire line found within the input field was not able to have any individual address components be parsed or identified.

An example of these two concepts would be:

Input:

WANBURY LIMITED

BSEL TECHPARK

B1/42/44 SECTOR 6 LANE

OPP. VASHI RAILWAY STATION

VASHI

NAVI MUMBAI

400705

IN

Generated Global Address Cleanse Output:

• COUNTRY: India
• LOCALITY1: Navi Mumbai
• LOCALITY2: Vashi
• REGION: MH
• POSTCODE: 400703
• PRIMARY_NAME: Sector 6
• PRIMARY_NUMBER: B1/42/44
• FIRM: Wanbury Limited
• POINT OF REFERENCE: Opp. Vashi Railway Station
• ADDRESS_LINE_REMINDER1: LANE
• EXTRA1: BSEL TECHPARK

Informational Codes (INFO_CODE)

The global address cleanse transform has a few selected output fields that will describe the cleansing process.  One in particular is called the INFO_CODE.   ASSIGNMENT_LEVEL and ASSIGNMENT_TYPE can also be used – more information can be found in the documentation at http://help.sap.com.

The INFO_CODE field is populated with a 4 digit number: 3010, 3000, 3070, 2020, etc.  This field can be analyzed and any necessary post-processing specific to your custom solution can also be performed on the output data if needed.

Hi Everyone,

SAP Early Knowledge Transfer for SAP Data Services 4.2 SP05: Learning Map for Solution & Technology Consultants is now available inSAP Learning Hub.

After you have the access to SAP Learning Hub with you will be able to see the followingSAP Data Services 4.2 Learning Map  (Search : EKT_BODS42SOLTECH)

• Overview
• Data Services Workbench
• Real-time Data Integration
• Data Services Transform Enhancements
• Data Quality Transform Enhancements
• TDP Transform Enhancements
• Data Services Job Lifecycle Management
• Other Data Services Enhancements
• Delta Knowledge on SP01 - SP04
• Delta Knowledge on SP05

  • Data Services 4.2 - What's new With SP05

  • File Location Object and Built-in Functions

  • MongoDB and Data Services

If you don't have the subscription yet, please go to SAP Learning Hub, choose your region and find the Local Education Contact in the bottom of the page.

See also:  Discover SAP Learning Hub

Hi All,

I have described the steps for loading Hierarchy from CSV to BW using BODS.

The Structure of the CSV file will be as shown below :

Plant is the Leaf nodes for which we are going to load the Hierarchy.  Also we need to have the text for each element which will be another CSV as below.

The object in the above text file structure will represent the info Object names in BW.

The File output will be like below

The overall job will look like below with 3 dataflow objects each performing a different role.

DataFlow 1 : CreatingNodeID

Query : (Remove the blanks value with updating the previous value)

The first Column country and state has blank values as shown above so we need to repeat the value till a new value in the column has encountered like in case of State AP should be repeated twice till new value KERALA has occurred in the data.

We do this using a custom function

The Custom Function DF_SAVESTR(Plant.Field1) will look like

IF input is not null the value in the column country will be stored in $Country. When the next value is null in the column the $input1 is not assigned i.e. null value is not assigned and the previous value in the $country returns nothing but the same value as INDIA for column country in our case.

The country column get the value in $country when the value in the file is null as below

Similar way we create the values for State column and the output of Query will look like

Then in order to get all the nodes in one single column we split the Query into four others Query Transforms as Country, State, City and Plant as shown in the data flow screenshot and also add a group number to each query transform to arrange the nodes in sequence.

Perform distinct selection to ignore the duplicates

Then perform a merge of all the individual transforms mentioned above to get all the values in a single column.

Merge:

Then perform order by based on GrpNum and generate the NodeID in the Final Query transform NodeID

The output of NodeID will look like below and save this in a table Master.

Master :

To get the Parent Child relation between the node we have another flow from Query which is split into 3 other transforms as Parent_child relation transforms like Country_state, State_city and City_plant.

The main purpose of these transforms is to generate the parent ID for each node of the File.

After we perform a merge on these three transforms the output will look like

Save the output in another table like Parent_child as shown above.

DataFlow 2 : CreatingMaster

This data flow works on retrieving the NodeID for parent nodes in the Parent_child table with lookup from Master table.

LookupNodeID :

We perform a lookup into master table referring the Parent Node with the NodeName of master the get the NodeId from the master table as shown in above screenshot using Lookup_ext().

The output  looks like: stored in table Master_Temp.

DataFlow 3 : ParentID and text (To add the parentID and text for each Node)

Get_ParentID:

We retrieve the parentID from Maser_temp table referring the NodeName from master table in above query transform with the Child in Maser_temp table and get the corresponding parentID for all the nodes in master.

Root_NodeID Transform : (Set the root node ParentID to ‘0’)

Here as we retrieved parentID for all the nodes in master table in the previous query transform, but the root node i.e. INDIA in our case will not have any parent and the value will be set to null in previous query transform.

To set to “0” we use this query transform.

Getting_NodeID :

We create the NodeID for the text item coming from the text CSV with reference to Master table and get the node , object name and text from the file.

Set_Object : Join the text and master data using NodeID.

The below two sets are joined

Output from Root_NodeID query transform:

Output from Getting_NodeIDquery transform:

The final output that we get is as below:

This can be mapped to the BW data source and loaded to BW.

Thanks,

Madhusudhan K.

Why do we need substitution parameters in excel?

In designer, we see substitution parameters in a grid view.

Whereas, when we have to export it, we will only have XML and ATL as option. These are not the straight forward information for humans to understand.

If there is a wider audience who wants to take a look at substitution parameters, instead of allowing everyone to login to designer, You can email them the substitution parameters in excel file

Idea behind the approach

Plan was to create a utility to export substitution parameters to CSV file from the repository. VB-Script was the easy way we could think of, as we were using Windows machine. Repository databases are hosted on SQL server.

Idea was to read repository database credentials from user. Export substitution parameters to XML file through al_engine.exe, and then convert it to CSV file.

Issues with comma separated values

• If there is a comma in SP value, cell value gets to split and span to multiple columns in excel. Tab separator was ideal.
• Date value will automatically undergo format change in excel upon opening the file. Cell value has been formatted as text.

VB-Script Code:

' Don't worry if you don't understand. Just copy paste the code in notepad, save it with vbs as extension and double click

' Or download it from attachment.

Option Explicit

Dim SQLHost, SQLDB, SQLUN, SQLPWD   

SQLHost = InputBox ("Enter target SQL Host,port:", "Export SP to tab delimited text file","")

SQLDB = InputBox ("Enter target SQL database:", "Export SP to tab delimited text file","")

SQLUN = InputBox ("Enter target SQL username:", "Export SP to tab delimited text file","")

SQLPWD = InputBox ("Enter target SQL password:", "Export SP to tab delimited text file","")

build_and_execute_command

SP_XML_to_CSV "SP.xml", "SP.txt"

Msgbox "Open generated tab delimited text file SP.txt in Excel." & vbCrLf & "If required, format it as table with header.",  vbInformation  ,"Export SP to tab delimited text file"

Function build_and_execute_command()

    Dim command, objShell, filesys

    set filesys=CreateObject("Scripting.FileSystemObject")

    Set objShell = WScript.CreateObject ("WScript.shell")

    command = """%LINK_DIR%\bin\al_engine.exe"" -NMicrosoft_SQL_Server -passphraseATL -z""" & "SP_error.log"" -U" & SQLUN & " -P" & SQLPWD & " -S" & SQLHost & " -Q" & SQLDB & " -XX@" & "v" & "@""" & "SP.xml"""

    export_execution_command "%LINK_DIR%\log\", "SP",command

    'objShell.run "%LINK_DIR%\log\" & "SP" & ".bat",0,true

    objShell.run "SP.bat",0,true

    filesys.DeleteFile "SP.bat", true

    if filesys.FileExists("SP_error.log") then

        msgbox ("Encountered issue while exporting SP from repo")

        build_and_execute_command = -1

    End if

    Set filesys = Nothing

End Function

Function export_execution_command(FilePath, FileName, FileContent)

    Dim objFSO, objFile, outFile

    Set objFSO=CreateObject("Scripting.FileSystemObject")

    'outFile = FilePath & FileName & ".bat"

    outFile = FileName & ".bat"

    Set objFile = objFSO.CreateTextFile(outFile,True)

    objFile.Write FileContent & vbCrLf

    objFile.Close

    export_execution_command = 0

End Function

Function SP_XML_to_CSV (xmlFile, csvFile)

    Dim ConfigList, SubParamList, objXMLDoc, Root, Config, SubParam, Matrix(1000,50)

    Dim i, j, iMax, jMax, Text, sessionFSO, OutFile, objShell

    Set sessionFSO = CreateObject("Scripting.FileSystemObject")

    Set OutFile    = sessionFSO.CreateTextFile(csvFile, 1)

    Set objShell = WScript.CreateObject ("WScript.shell")

    Set objXMLDoc = CreateObject("Microsoft.XMLDOM")

    objXMLDoc.async = False

    objXMLDoc.load(xmlFile)

    Set ConfigList = objXMLDoc.documentElement.getElementsByTagName("SVConfiguration")

    i = 1

    Matrix(0,0) = "Substitution Parameter"

    For Each Config In ConfigList

        Set SubParamList = Config.getElementsByTagName("SubVar")

        j = 1

        Matrix(0,i) = Config.getAttribute("name")

        For Each SubParam In SubParamList

            If i = 1 Then Matrix(j,0) = SubParam.getAttribute("name")

            Matrix(j,i) = "=""" & SubParam.text & """"

            j = j + 1

        Next

        i = i + 1

    Next

    iMax=i

    jMax=j

    For i=0 to jMax-1

        Text = ""

        For j=0 to iMax-1

            Text = Text & Matrix(i,j) & vbTab

        Next

        OutFile.WriteLine Text

    Next

    OutFile.Close

End Function

Usage screenshots:

In Excel, open the text file:

Select all the data cells and format it as table

Finally, Data looks like this:

If you don't have access to repository database or Jobserver, you can export substitution parameters to XML file manually from designer and use the function SP_XML_to_CSV from the given VB-Script.

Cheers

SAP Data Services' HistoryPreserving transform does a good job to reduce complexity of history preserving when loading data into a data warehouse. However, it has the limitation that ValidFrom and ValidTo columns can only be a date - not a timestamp. So to allow for history preserving of intra-day changes, we need a workaround. In this blog post I'm showing how this can be achieved by using the Map_Operation transform multiple times:

The dataflow shown above loads table CUSTOMER to table CUSTOMERHIST. CUSTOMER has 3 columns (PK INTEGER, FirstName VARCHAR(100), LastName VARCHAR(100)) with PK being the primary key. CUSTOMERHIST has two more columns ValidFrom, ValidTo both of type TIMESTAMP; Its primary key is (PK, ValidFrom). We also need to set a variable $CurDateTime at the beginning of the job to use the exact same timestamp in UPDATEs and INSERTs:

$CurDateTime = concat_date_time(sysdate(), systime());

The TableComparison transform looks up incoming records in CUSTOMERHIST that have the same value in field PK and where ValidTo is null (compare to current records only). In this example we also enable "Detect deleted row(s) from comparison table".

TableComparison outputs rows of type INSERT, UPDATE and DELETE. We are multiplying this output and send it to Map_Operation transforms: one for INSERTs (I2I), two for UPDATEs (U2U, U2I) and one for DELETEs (D2U). In the "Map Operation" tab of each Map_Operation transform we configure the output type of the record; we discard the records that are handled by the other Map_Operation transforms:

Now we set the column mapping for each case:

I2I:

If TableComparision sends an INSERT, there's not much todo. We keep the values as they are and just set the ValidFrom column to $CurDateTime

U2U:

If TableComparision sends an UPDATE, the first action is to set ValidTo to $CurDateTime since this record is not the current one anymore. We want to keep FirstName, LastName as it is in CUSTOMERHIST and do not overwrite it with the new values from CUSTOMER, so we need to map FirstName, LastName to before_image(FirstName), before_image(LastName), resp.

U2I:

If TableComparision sends an UPDATE, the second action is to insert a new record with the current values from CUSTOMER. As I2I above, we just need to set ValidFromto $CurDateTime. Note that we need to change the update section (middle), not the insert/normal section (left).

D2U:

If TableComparision sends a DELETE, we need to update the current record by setting ValidTo to $CurDateTime.

With this configuration, the four Map_Operation transforms together replace one HistoryPreserving transform. The example does not include an ISCURRENT column but it should be straight forward to do this enhancement. If there is a generated key column in CUSTOMERHIST, this could be populated using the KeyGeneration transform after merging U2I and I2I. The picture below show the status of both tables after inserting, updating deleting a record in table CUSTOMER:

History preserving after INSERT of record 4711:

History preserving after UPDATE of FirstName to 'John':

History preserving after UPDATE of FirstName to 'John D.':

History preserving after DELETE of record 4711:

We can add the jobs under projects by dragging them into project folder(From object library to project area)

But if we have hundreds of jobs its painful to add each job by dragging into project folder.

so by using below steps we can add the jobs under projects at a time if we have many jobs also.

Step 1 :

Take the list of job details in excel(Which you want to add under project) and remove duplicates and append comma(,) end of every row(end of job name in every row) except last row,append semi colon (;)end of last row.

Step 2 :

Export the Project under which you want to add job names.

Step 3:

Edit the exported ATL in notepad and add the job names in ATL and save it.

Step 4: Import saved ATL into Repository and open the project you will get the new jobs under project.

We can save the time by using this method to add hundreds of jobs under project.

Overview

The proliferation of JSON data available from web service based sources is enormous these days. So I wanted to capture some JSON web service data for a proof of concept via SAP Data Services, but was discouraged to find there was no simple way to do this.

Searching for a solution, there were a couple of posts that vaguely refer to using the Data Service Adapter SDK to build a special adapter.  But this seemed like a complicated route for something so simple as JSON.   However then I found out about Base_UserDefined transform.

In this article, I will use Base_UserDefined transform to return web service data using the “Per Collection” option.  I initially learned about Base_UserDefined from Kamal’s excellent article, http://scn.sap.com/community/data-services/blog/2013/01/31/how-to-use-userdefined-transform-in-sap-bodsbusiness-object-data-services.  But in that article it uses Base_UserDefined in a “Per record” mode.  My json request will return many records as a result so I need to use the “Per Collection” mode and it requires a different way to work with the Collection object of Base_UserDefined.  I could not find any articles or documentation on the "Per Collection" mode so I will help fill that gap with this article.

The Base_UserDefined transform simply provides an interface to the Python engine and language.   There is also a strong community of Python code one can get assistance with.

By the way, Python is an object-oriented programing language that is interpreted making it a very quick development tool.  But what is most helpful are the module and packages that are supported by an open source community.  It is rare that I do not find a module already out there already is written for the job I need to do.

How it’s done

In this example, I want to return data from a json data source maintained by the Consumer Financial Protection Bureau (my daughter is just starting college and the number of credit card offers she is already receiving has caused me to think “financial protection”).  Hense the choice of web service.

The url I will use is this: http://data.consumerfinance.gov/api/views.json

1. Prerequisite - First you will need to download and copy the urllib2.py module to your \Program Files (x86)\SAP BusinessObjects\Data Services\DataQuality\python\lib\ folder. This should be on the server the job server is running on.
   
   urllib2.py is a module that handles web service requests.  You can find it here and then save it as a file called urllib2.py.
   http://svn.python.org/projects/python/branches/release22-maint/Lib/urllib2.py
   
2. Now to the design work.  We need to create a source that provides the URL for the web service.  In my example I simply added a SQL Transform and wrote a SQL select statement with hardcoded url.  
   Select statement I used is
   select 'http://data.consumerfinance.gov/api/views.json' as JSONuri
3. Next drag in to the Data Flow a Base_UserDefined transform from the Data Quality\User Defined folder in Transforms, add a Temp Table, and connect all three.
   
   
   
4. Open the Base_UserDefined transform and in the Input tab of the transform, drag and drop Input column of the input schema to the Input Schema Column Name so as to map with the Transform Input Field Name.
   
5. Now click on the Options tab and then click on Edit Options button.
   
   
   
6. Select the Mode as Per Collection (this is key, one input record will create multiple output records) and then click on Python Expression Editor to display editor on the right.  You can paste the code directly in this window for now but we need to setup output fields via the “Launch Python Editor” button on the right.
   
     See get_json.py.zip  attachment for the python code.
   
   
7. Notice there are thee tabs on left panel.  Click on the I/O Fields and then right click on Output Fields to select Insert.  Do this to create two output fields called complaint_id and complaint_name both as varchar 50.
   
8. Click Ok twice to close the editor and then click on the Output tab.  From here you simply click on the check box next to each output field which will result in their mapping on the right panel. (I set complaint_id as a primary field but that is not required.  Just me being picky.)
   
9. Now launch the job and you should see a result in template table like this:
   

Conclusion

Using Base_UserDefined transform opens up a whole new world of flexibility and productivity but leveraging Python.  In this case I did not need to write any code to handle the web service request or to parse json and now I can load a database of all US Consumer Complaint departments.

Tip:  It is tricky to debug a Python script within a Base_UserDefined transform, so I will often create a plain version of the script and run it from my own Python engine and then convert it to one that will work in Data Services.   I will often use a Python Print statement to display variables values which you will be able to see in the DS job log.

The Case Transform is used to route one row to different outputs depending on conditions. Specifies multiple paths in a single transform (different rows are processed in different ways).

The Case transform simplifies branch logic in data flows by consolidating case or decision making logic in one transform. Paths are defined in an expression table.

Here I have taken an example EMPLOYEE table.

Employee Location are 3 coutries - IN,SG and US.

Requirement:

To load the LOCATION data into 4 different tables.

If LOCATION=IN then load the data into IN_EMP

If LOCATION=SG then load the data into SG_EMP

If LOCATION=US then load the data into US_EMP

If LOCATION is other than IN,SG and US then load the data into OTHER_EMP

Bring the Case Transform after the Query Transform as shown below:

Inside the Case Transform use the code as shown in below screen.

Run the Job to load the data as per our Case transform logic and check.

SAP BODS 4.2 has new transformation called “Data Mask” ,which is used toenables the data masking
functionality .

Eg: Protect personally identifications ids (like SSN, PAN, etc..) and information in your data. Personal information includes data such as credit card numbers,
sal information, dob, phone numbers, credit card numbers or bank account numbers. You may want to use data masking to support security and privacy
policies, and to protect your customer or employee information from possible missuses.

I have created a source table with data as looks below.

After masking the results would be

This will explain the steps to mask the personal information  like ID,CC and EMAIL.

SAP BODS Data Mask Transformation:

Created a Data flow  with source-->Data Mask-->Target mapping.

Data Mask Transformation Properties :

Data Mask transform has 3 tabs 1.Input 2.Option 3.Output tabs

Input tab properties:–In input tab map the respective source columns accordingly

E.g: Mapped input accordingly...

ID-->GENARLA_DATA1

CC-->NUMARIC_DATA1 (as CC has numeric values and datatype is int)

EMAIL-->EMAIL1

LOC-->GENARLA_DATA1

Options tab properties: option tab allows to specify the masking options for each column and create as many as “mask out “by right click
and create duplicate Option.

Column assignment and specify the masking characters like as follows below

• Map the input Field –EMAIL (Browse the column name where you want mask)
• Starting position –END/START (means how you want to mask from start or end
• UnmaskedLength—length (how many characters you want to mask)
• Masking Character –Which character you want to use for masking

Perform the same steps for all required columns.

Output properties : Check the requires columns in output tab

Results:

Custom scheduling  without “.BAT” file or any third party schedulers:

Introduction:

This Article provides solutions for custom scheduling BODS Batch Job conditionally. As you are aware that BODS does not contain an inbuilt mechanism to customize the schedule of a job. However, we cannot execute workflow on its own and need a Job to execute it and in various scenarios there is a need to sequence the Jobs and conditionally run the Jobs. The approaches provided below can be used where we can customize our scheduling as per the requirement and without using any .BAT files or any third party schedulers.

• With this approach we do not need any third party softwares or any scheduling tools.
• With this approach we do not need any BAT files or SH scripts or any LINUX environment.
• We can avoid scheduling the entire job and wasting the time and impact.
• Avoids loading a Job with too many Workflows.
• At any point, Child Jobs can be run independently too in Production Environment, this will not be possible if the entire Job logic is put into a Workflow.

We can schedule a job with .BAT file by using below procedure:

• Create a script in designer, inside script use below code:

Exec( 'cmd.exe’ , 'E:\\SAP_SCHEDULE\\JOB1.BAT',  8) ;   

Print(  ' Job_1 Completed....') ;

Our aim is to schedule a job without .BAT file:

Let us assume the requirement is to schedule a job on the third Friday of each quarter of the year.

Generally, to approach this requirement, we create a .BAT file and schedule it with EXEC command as shown above or we have to manually execute this job on particular date.

How to automate this kind of scenario?

In the following I will explain the process to achieve without .BAT file

I have created a Project: PRJ_CUSTOM_SCHEDULE

And Job: JOB_CUSTOM_SCHEDULE.

And then, I have created a script: SCR_THIRD_FRI

The script contains below logic:

          $G_SYSDATE = to_date( sysdate(), 'YYYY.MM.DD');  à

          #$G_SYSDATE = to_date( '2015.07.11', 'YYYY.MM.DD');

          print( '  $G_SYSDATE '||$G_SYSDATE); --> printing the date

          $G_THIRD_FRI = to_date (sql( 'DS_NAME' ,'select b.CALENDAR_DATE from ( select row_number() over (partition by YEAR_NUM, QUARTER_NUM order by                YEAR_NUM, QUARTER_NUM desc) as QTR_WEEK_NUM,  a.* from (  select CALENDAR_DATE, YEAR_NUM, QUARTER_NUM from SCHEMA1.DATE_DIM                where DAY_OF_WEEK_NUM = 6 and YEAR_NUM = YEAR(CURRENT_TIMESTAMP) and  QUARTER_NUM = DatePart(QUARTER, {$G_SYSDATE})  ) a) b                where b.QTR_WEEK_NUM = 3 '),'YYYY.MM.DD');

          #$G_THIRD_FRI = $G_SYSDATE; 

          Print (‘$G_THIRD_FRI: The job execution date is:   ' || $G_THIRD_FRI );

The query written in the script is calculating third Friday of each quarter. This query is written on Date dim table. Schema of Date dim might be different from project to project. You need to modify the query (eg: column names) as per the schema of your Date dim.

I joined this Script with a conditional object as follows:

Inside the conditional object:

The actual business logic(WF_BusinessLogic) works if the fetched/calculated date is equal to current system date or else the script(SCR_NOT_3FRI) will be executed which says today is not 3^rd Friday of this quarter.

The condition in the script make sure that the workflow will execute only on 3^rd Friday of each quarter.

Once this is done, Schedule the job JOB_CUSTOM_SCHEDULE

Login to Data services Management Console:

1. Go to Administrator tab -->

Select Batch under Administrator

Then -->

Schedule the job :

Advantages of this approach:

• Calculating job scheduling day is pushed down to database server.
• Maintaining the code is easy compared to .bat / .sh method. Can be reused as reusable component

Hello Readers

How do you usually delete objects from repository?

1. Navigate across tabs in "Local Object Library"
2. Visually identify object names
3. Hit delete button & click yes for delete confirmation

That becomes a tedious work if you have to delete hundreds of objects from thousands in the object library.

al_engine.exe became our super hero when we discovered that it can delete objects for us when provided with certain parameters.

Here is an example to delete a job with the name JOB1 which is in Microsoft SQL server repository:

"%Link_Dir%\bin\al_engine.exe" -NMicrosoft_SQL_Server -U<UserName> -P<Password> -S<HostName> -Q<databaseName> -XRJ@Job1

• Text in angular brackets are database logon information
• Red text is object type
• Blue text is object name

Based on the same pattern, command generator in excel can be built to delete many objects easily.

Formula in B9 is

="""%Link_Dir%\bin\al_engine.exe"" -NMicrosoft_SQL_Server -U" &  $B$3 &" -P" & $B$4 & " -S"&$B$1&" -Q" & $B$2 & " -XR" & $B$5 & "@" & A9

Object types can be any of the following

Object Name can be "datastore"."owner"."name" in case of objects contained in a datastore.

For example, table, stored procedure, domain, hierarchy, or IDOC.

Once commands are generated, we just have to copy all and paste it in command prompt.

Also note that the delete confirmation will not be asked. Object gets deleted as soon as command is executed.

Make sure you take repository backup before executing delete commands. Of course, al_engine can do it for you. Execute the below command to take complete repository back up in single ATL file "repo_export.atl" in %Link_Dir%\log directory.

"%Link_Dir%\bin\al_engine.exe" -NMicrosoft_SQL_Server -U<UserName> -P<Password> -S<HostName> -Q<databaseName> -X

Hope it helps you save your time and effort.

Cheers

my source is db2 and target is hana entire table successfully uploaded into target.but i got one error for one table migrating i.e intigrity unique constraint violation error.

from db2 we are getting few special characters into bods but bods converting into same format.that data becoming in hana  like duplicate.so that hana is not allowing the duplicate data.

so that iamgetting unique violated error .

i tried at source data store level i enable locale :utf-8 its running when only special characters available.but any int and char values is their  again its giving the error.

Thanks

Naga

Hi All,

Here I'm explaining the Propagate Column From feature of BODS. Say, you have created a job in BODS. Now due to change in requirement you need to add few more information. Then what is the way to do it more effectively in less time. Do you have to manually go through all of your Data Services ETL process elements and add those new columns? No. In BODS you can easily and very quickly achieve this goal by using Propagate Column From feature.

Here is how you can achieve this in BODS:

1.      Open the dataflow in which you would like to add new columns

2.      Open the data flow final transform where you would like to see your new columns as a result

3.      Right click on the tree root element and select Propagate Column From

4.      In the new opened pop-up window select the data source where those new columns exist

5.      Select the new columns and press Propagate

6.      The new columns are automatically added to every transform from the data source to the target

All newly added columns can be treated, changed, deleted and maintained as normal manually added columns.

Regards,

Gokul

In Data Services all the names of workflows or dataflows are cut off after 17 characters.

So to fix this go to Tools -> Options and click on General.

You should able to see : Number of characters in workspace name. Change the number 17 to 100. Click OK

Regards,

Tanvi

Hello all,

I came across a post by Arun Sasi regarding how to add an attachment through BODS via VBScript. The link to it is here, and I recommend you take a look to get an idea of the steps involved: http://scn.sap.com/docs/DOC-60700.

I found it very informative, and with the help of the comment by Santosh Panda on the same post, I was able to find a way to use a Custom Function to automate the procedure. To begin, let me mention why I decided to go about doing this.

1) The original post makes it clear that the attachment to send and the VBScript must be in the same file location

     a)This was problematic to implement for us, because that means each batch job would need a new script based on the file directory

     b) Our network locations get deleted every 10 days, and changing that to house a script on the network was out of the question

2) Ease of use

3) To eliminate some of the limitations I found in implementing a permanent file location with a script

     a) One such issue was the 'From' and 'To' parts of the email change for our Jobs based on the environment we are in

4) To disprove the 'BODS can't email attachments' notion

     a) Although technically, it is still true. We use VBScript to do it

5) For the fun of it

All, Apologies for the picture content. I do not know why they appear so grainy. If you click on them, they appear much better. Thanks!

The Method

Starting out, I had no direction, but I will give you the end result as opposed to the steps to achieve this on your own. The Custom Function will be our starting point, and does the majority of what we need in order to carry this out. Then, we will look at how to set up variables to pass into the script. The [minimal amount of] ETL required to make it work. And lastly, the exec() function to pull it all together. You will notice that I redact some portions of the scripts for privacy purposes, but will maintain the majority.

The Custom Function

This is CF_Email_Attachments. Aptly named, because it can email attachments. The First chunk of this, regarding $LV_Env can be ignored. It helps determine the mailing address we use for different environments. The main information required is actually after the Return statement, which is a single large string. We Set the Return Parameter Variable to Varchar of size 2048. The VBScript itself comes in the lines to follow, with pipes for concatenating portions of the string that I have used as variables. For documentation on the VBScript itself, I recommend a google search on how to send emails with VBScript, and you will see something quite similar.

As you can probably imagine, the more information you can store in the Custom Function to automate, then the fewer parameters that need to be passed in to make the function work. This makes usability easier, but sometimes different batch jobs require different parameters.

I will note that $PV_Attachment_Location_Name is the entire file path to the VBScript that we will declare in the script below.

The First Script

The first script is arguably the most important, and requires a set of things for this to all work.The placement of the script just needs to be before the use of the Custom Function. 

Not all of this is required, but some of it is necessary. Starting at the top, $LV_Integration_Name, $LV_Date_String, and $LV_File_Dir are not required, but for our uses were helpful. Using a date string in the name of a report allows a record of that report for future use.

$LV_Processed_Location is the first required step. This is the location we intend to save the VBScript that the Custom Function generates. $LV_Test_Report_FileName is step two, as I am assuming that you would like to send a report that you generated via BODS. Save a flat file as a .csv, or name it whatever document type you like.

$LV_VBS_Automation is the name of the VBScript. Notice how we save it as a .vbs. This is important.

The next 4 Local Variables are the 4 parameters that the Custom Function will use to generate the EmailUser.vbs.

Generate the file you wish to send

I am not going to go into this, but at this stage, you should have the ETL to generate a report or file that you wish to send, however that looks.

The ETL

Assuming you created the report in a data flow already, we need one more data flow to create the VBScript. It does not (and should not) be complex. 

Qry_Automate:

Notice how I have Select Distinct turned on. This is extremely important, as BODS will generate tons of scripts if it is not, and will send potentially hundreds of emails. Also, notice how I needed a source table, even though I am not using anything from it, this is a BODS limitation.

The name of the column really does not matter.

Target file:

This is nothing special, except I should note the data field for the column VBScript in the mapping is also set to Varchar 2048. This is the file we named as EmailUser.vbs in the first script.

So far, we have declared the variables we need, generated a report, used the Custom Function in a mapping to a file we named with a .vbs suffix, and now we need to use it all.

The Magic:

The magic happens in the script after everything else has ran. The exec() command is quite marvelous, and can be used to execute just about any file. For us, however, it looks like this:

That is it. Once the job runs, it will initialize our parameters, generate the file to send, map the field using the Custom Function, save it into a .vbs script and execute it with exec() command. Given that you are apart of the mailing list you emailed to(or if you emailed it to yourself) you should receive an email!

Thanks again to Arun Sasi for the original post and Santosh Panda for providing some more details. It was a fun journey getting this all worked out, and I hope I provided you and your teams a new capability!

Here's a custom function that converts decimal to binary.Feel free to use it and let me know if you find a bug when you use it.It is developed and tested in SAP Data Services 4.2

#input => base 10
$L_next_no = $G_Decimal_input ;
$L_bin_add ='';
$L_bin ='';
while ( $L_next_no  >= 1.0)
begin  $L_bin = $L_next_no % 2;  $L_bin_add = $L_bin_add||''||nvl($L_bin,0) ;  $L_next_no = $L_next_no / 2;
end
#Reverse
$L_len = length($L_bin_add );
$L_ret = '';
$L_CNT = 0;
while ($L_CNT < $L_len )
begin   $L_CNT = $L_CNT + 1;   $L_ret = substr($L_bin_add , $L_CNT, 1) || $L_ret;
end
return($L_ret);

Variables declared

     $L_ret                VARCHAR(255)     $L_next_no      VARCHAR(255) ;     $L_bin_add     VARCHAR(255) ;     $L_bin                VARCHAR(255) ;     $L_len                INT ;     $L_Ret               VARCHAR(255) ;  (Output)     $L_cnt                INT ;    $G_Decimal_input VARCHAR(255); (Input)

_no = $G_Decimal_

Call

print(cf_convert_decimal_to_binary('2684110470'));