Blog

Audit finding: productive system is modifiable!

Shortcut IT — Mon, 07 Oct 2024 06:04:00 GMT

It is a standard check done by many auditors to have a look into the system change options. And even if the admins did not open the system changeability or any component, sometimes there is at least one component open for modifications! How could this happen?

The reason for this often is in implementing a new component with the SAINT transaction, which leaves the freshly installed component with status 'modifiable'. Also a 3rd party transport request (or from SAP) could cause this situation.

This likely becomes a finding.

This finding can be easily avoided by implementing an automatic closing of the global setting, the components and the namespaces.

Unfortunately there is no SAP standard program available that can be scheduled daily. In that cases, this is where usually the idea comes into play of developing an ABAP program for this purpose, distributing it to all systems and scheduling it as a periodic batch job in all productive systems.

It can be shorter, faster and consequently cheaper. "Shortcut for SAP systems" comes with a 'Update table data' function, and with the command line tool we can automate it.

First let's have a look at the SAP function for setting the system changeability (SE03) and determine which tables are to be updated:

These are the tables behind the 3 categories of the system changeability:

The changeability of the "Global Setting" (1) is stored in a single TADIR table entry:

The changeability of the Software Components (2) is stored in the table DLV_SYSTC:

And finally the changeability of the namespaces (3) is stored in table TRNSPACE:

As usually for this kind of tasks, the command line tool of "Shortcut for SAP systems" is useful. It runs outside the GUI application on OS level. This gives us the possibility to execute it periodically via an automation solution. This can be a simple (and free) solution like the Windows Task Scheduler or a more sophisticated (and more expensive) product like Automic Automation or others.

There are 2 options to use the command line tool:

using a Variant (a Variant in "Shortcut for SAP systems")
using an XML file

As it is here about an update of 3 tables, the 2nd option - using an XML file - is more suitable. The 1st option would require 3 variants and 3 single calls of the command line tool. With the XML file we can use a single XML file with 3 update tasks and a single call of the command line tool.

An XML file for updating the table entries would look like this:

For each of the settings (Global Setting, Software components, Namespaces) we update the records in case they do not already have the right setting (that is where the tag is specified for).

In front of scheduling this periodically we call it directly on the console:

A verification in the system (SE03) showed that it worked as intended. In case the settings are already correct, the return code would be 4 - a warning, because the update did not apply on any record.

Now let's implement this in a periodic task. For this have a look into this article (which is about a very similar issue).

With the implementation of a periodic run you can be sure, that system changeability in next year's auditing will not be an issue anymore. We did not have to develop an ABAP program and distribute it into every system line, followed by a job scheduling. Adding a new system for this task is quite simple via copying the lines and just use another connection for the copied lines.

Speed up the system refresh

Shortcut IT — Tue, 27 Aug 2024 09:30:00 GMT

Doing system refreshes of your quality / test systems is important for their quality and usability. But it could turn into a big effort, and additionally there is time pressure. After all, the systems should be available again quickly. With "Shortcut for SAP systems" and the information from the "PCA" tool, the pre- and post-processing can be automated. Read here, how you can speed up processing the data during pre- and post-processing significantly.

This article is a continuation of this one. So if doing a system refresh with "Shortcut for SAP systems" is new to you, you should first read this article.

The topic in this article is the speed-up of the pre- and post-processing. With this you can significantly reduce the runtime of the pre- and post-processing of the system refresh!

There are 2 starting points:

1.: Between the several components are intersections of the tables. Have a look for example at components SWU3 and SWU3_DATA: you will find lots of tables that are part of both components. Or components BSSUSER and USER: all tables of BSSUSER are also part of component USER. Consequently, lots of tables will be exported / imported twice! Regarding optimization of the runtime a single export/import of these tables is to be preferred.

2.: Use parallelization! Here we have 2 possibilities:

By using a single XML file for the export and import tasks, task by task in the XML file will be processed sequentially - the next task will be started after the previous task has been finished.
But usually we have a lot of resources in the SAP system available when doing the export or import. There are no users or batch jobs active on the system in these phases. So just leverage the system by using some more work processes.
From the PCA tool we get the information of all components and their tables, and using the program supplied with our product we also get the information about the table size. Let's use this information to divide export and import into pieces - balanced by using the size information - and start them in parallel.
Also R3trans offers the possibility to use parallelism at the import - big files will be splitted into portions and processed by child processes of R3trans. This usually reduces the runtime of an import with R3trans significantly.

With the recent version of "Shortcut for SAP systems" and the program supplied with our product (ShowPCATables.txt in the ABAP folder, or downloadable here) the realization of the parallelism - by using multiple work processes as well as using R3trans parallelism - becomes quite easy! Let's make a walkthrough.

We start the program and select the "Refresh" option. Note: for a good balancing of the parallel tasks, we need the size data of the tables. The program should therefore be started on the target system of the system refresh.

By default the PCA tool lists components NRIV and SNRO, both contain table NRIV (number ranges) with its full content. As we are going to do a system refresh from our productive system including all business documents, we take also the number ranges from the productive system and leave this table out here (1).

The selection screen accepts some information that is written to the XML files. We create XML files for a system refresh (2) and specify the connection (3) we want to use. It is convenient to have all data and log files in a dedicated directory (4), which we also specify here.

And now it's time to optimize performance:

With activation of the flag "Avoid multiple export/import of the same table" (5) a table will be exported / imported only once - although it might be part of multiple components. So with this we cover the 1st of the above mentioned starting points to reduce the runtime.

This requires that for export and import the same components will be processed! If a table t is part in components compA and compB, for one of these components the table t will not be exported! So the data file of one of the components will be incomplete. If lateron both components will be imported, this does not matter, as table t will be imported with one of these components!

The 2nd above mentioned starting point will be realized by specifying the amount of parallel tasks (6).
On my small test system I set the value to 3 - means, for export as well as for import the tasks will be distributed to 3 files, balanced as good as possible based on the data size of the components. These 3 XML files can be processed in parallel. Each execution occupies a dialog work process.
Feel free to leverage the system with more tasks than 3. However, you should not use all available DIA work processes of your system, finally few work processes should be reserved for being able to have a look into the system.
If there are components with a huge amount of data, the balance regarding the data volume (and also the runtime) might become less optimized. Think of a single component with 100 GB data volume, whereas all other components in sum have 20 GB data volume only. The export and import of this single component would dominate the runtime. When the job has been finished for all other components, this single component would still be in process.
With a flag at "Split big components..." (7) such a big component can be splitted into several parts and with this a better balancing of the parallel tasks can be achieved.
You can apply this splitting option also to the USER component. But the USER component is somehow special: for this component some "shadow tables" are used for export and import, and these shadow tables require a special step in front of the export (filling the shadow tables) and a special step after the import (processing the shadow tables and adjusting its data with the user tables). Therefore this is also a special option here. If you use it, you have to consider 2 additional XML files (will be generated if this flag is set), where this shadow table processing is implemented. In this example we leave this out, because in our system the data volume of the USER component is not that big. But for systems with lots of users this is a further option to reduce the runtime in case the data volume of the USER component dominates the others.
With the input field (8) we can use the parallelism option of R3trans.
The best amount of parallel child processes of R3trans depends on CPU's, database capacity, I/O, location of the database server etc.
You can find more details in SAP note 1127194. There is a rule of thumb given: setting the parallel value in the range of 1 to 2 times the number of CPUs results in the highest performance gains.
And finally we have to consider, that export and import are running in DIA work processes. Usually there is a time limit set for DIA work processes, ensuring that long-running programs do not endlessly block dialog processes (profile parameters rdisp/max_wprun_time and rdisp/scheduler/prio_.../max_runtime).
Here (9) we have the option to extend this maximum runtime temporarily. It can be set to 3 hours or the time limit can be completely switched off.
Activating this option leads to either...
- further tasks in the XML file in case you do not use parallel tasks (6)
or
- additonal XML files in case you use parallel tasks.
As this article is about optimization of a system refresh, of course we use parallel tasks and therefore we have to consider the processing of 2 additonal XML files.

After clicking on "Execute" we will get the list of the tables according to the PCA tool and also some size information. Feel free to use the possibility of setting filters to exclude components you want to get from the source system of the system refresh, so these components are not to be exported and imported lateron. You can use the ALV layouts to store the filter settings so that you do not have to set the filters every time you use the program.

After clicking on the "XML file" button we will be asked for the location for the generated XML files. In this example we already have prepared a directory with 3 directories inside for the single phases.

The program now stores 8 XML files in the directory:

3 for the export (as we specified 3 parallel tasks)
3 for the import
1 file for extending the maximum WP runtime
1 file for setting the maximum WP runtime back to default

The components with their tables were distributed to 3 XML files, with consideration of their size - as the size is an indicator for the runtime approximately needed. For optimizing the runtime, the target is to start 3 tasks in the system with similar workload. If all 3 tasks need nearly the same runtime - and finish at nearly the same time - the balancing is quite good.

After the XML file generation the program gives some summarized information about the content.

As we used the option for avoiding multiple exports/imports of the same tables, we now have a smaller amount of tables in comparison to the ALV list of the tables, which has 166 more lines!

The XML files contain the export and import stuff for each component.

The other 2 generated XML files are dealing with the maximum runtime for DIA WP's. One for extending the runtime - to be executed in front of the export tasks - and one for resetting the maximum runtime to its default value after processing the data.

Of course, in front of the export some preparations are to be done. Beside the planning and announcement, before starting the export the users are to be locked and logged off, batch jobs are to be suspended etc. This has to be done in advance, and we can do this similar to the export - using the command line tool with an XML file. You will find an example for this in the "XML" folder of our product.

Since we now use tasks in parallel, the start of the pre-processing (including user locking, suspending batch jobs, etc., and the 3 export tasks) is no longer just a single call to the command-line tool "Shortcut for SAP systems".

(1) First we consider the XML file for extending the maximum WP runtime.

(2) Then we lock the user, kick them out of the system and suspend the batch jobs.

(3) After finishing this the 3 export tasks are to be started in parallel.

This makes it a little bit more complicated. Starting tasks in parallel on OS level is not a big issue on a Windows computer, but we also want to know when the tasks have been finished. Thinking about automation, after finishing the export of all data, the system copy tasks - including the database copy - are to be started. Ideally also automatically!

(4) At the end we revert the maximum WP runtime back to its default.

I copied the generated XML files for the Export into a directory for the prework and renamed them with "Export_task1.xml", "Export_task2.xml" and "Export_task3.xml".

Have a look at this command file "main.cmd", executing the prework of the system refresh, which uses a Powershell script that controls start and end of the parallel tasks:

After the prework (extending maximum work process runtime, user locking etc.), a Powershell script "StartTasksAndWait.ps1" is executed. You can find this script as part of all files used in this article here or supplied with our product (folder xml, file "Example_Systemrefresh_runtime_optimized.zip".

It works as following:

it reads a file "ParallelTasks.txt" that contains the calls of the command line tool with the XML files for the export
for each line an asynchronous task is started
it periodically checks (default: every 5 seconds) whether the tasks are still active
after all tasks have been finished, the Powershell script ends.

So in addition to this we have to fill the file "ParallelTasks.txt":

The single command files for the export tasks are just calls of the command line tool with the XML files for the export:

It may seem a bit complicated, but it is not. Once setup, you can use it again and again.

Finally we have a single command file, executable on OS level, automatable with any suitable tool, doing the complete prework including user locking, suspending batch jobs etc. and executing the export in parallel tasks in a balanced manner, saving much runtime!

After starting main.cmd the procedure starts, and if you arrange the 4 raising console windows, it will look similar to this:

The upper left window (1) shows the console for "main.cmd", the other windows are the 3 tasks for the export, started in parallel. In window (1) the tasks (windows (2), (3), (4)) will be checked perodically. Once all tasks have been finished the prework of the system refresh is done - we saved much time by not exporting tables redundantly and by using parallel tasks!

When running command files in the Windows console, you may have sometimes noticed that the script strangely hangs until you press the Enter or Esc key.
The reason for this is a click into the console window (often done by mistake) in combination with a so-called "Quick Edit" mode. Find some details for example here.
When clicking into the window, the Quick Edit mode stops further outputs to the console window. If the command line tool is already running the tasks of an XML file, it does not have an impact on the command line tool's work. However, it blocks the script from continuing after the execution of the command line tool.
To avoid this, the Quick Edit mode should be deactivated, at least temporarily. There are several methods to do this:
- execute the command prompt as Administrator, then right-click on the title bar, choose "Properties" and deactivate "Quick Edit mode" in the options dialog.
- change the value for HKEY_CURRENT_USER/Console/QuickEdit to 0 using the Registry Editor (regedit).
- change the value for HKEY_CURRENT_USER/Console/QuickEdit to 0 by using this command:

reg.exe add HKCU\Console /v QuickEdit /t REG_DWORD /d 0 /f

If you like to stuck on the Quick Edit mode, you can add the "reg.exe add HKCU...0 /f" command to the "main.cmd" file at the start for disabling Quick Edit and the same with "...1 /f" at the end for activating it again.

For the post-processing phase it is the same. Especially for the import phase we will save a lot of time with the combination of parallel tasks (using multiple work processes in the SAP system) and the parallelism option for R3trans. It could not go any faster!

Find all files necessary for the pre- and postprocessing of a system refresh supplied with our product (folder xml, file "Example_Systemrefresh_runtime_optimized.zip"), easily adoptable to your needs. Alternatively you can download the ZIP file here.

Stop the mess in your IDoc processing

Shortcut IT — Tue, 06 Aug 2024 12:00:00 GMT

Are the IDocs in your system piling up into an ever-increasing chaos? Is your database constantly growing and is it taking longer and longer to process incoming IDocs or to list them in BD87 and similar transactions?

Read here how to clear up this mess.

Recently I saw it again. The IDoc tables... what a mess!

Millions of IDocs that could be archived - still remaining in the system.
Lots of IDocs, queued into processing again and again, although they were created years ago.

In the observed SAP system, the IDoc tables occupied ~700 GB in the database (without the indexes). Of course, these ~700 GB were copied to the quality system and another test system as part of a system refresh, so we are faced with a multiplied wastage of space. And as you can imagine, this is not only a waste of storage space, but also of time and money.

But more than this waste of resources and money, I think that constantly processing - or attempting to process - years old IDocs is unnecessary and dangerous. The hit list of these IDocs was led by an Idoc with ~230000 status records. It was created ~8.5 years ago, and from that point in time up to 7.5 years the system tried to process this Idoc again and again. Approx. 1 year ago, someone took pity on the IDoc and set it to status '68' so that no further processing attempts were made.

How much CPU capacity was wasted on this ancient Idoc during this time?

This IDoc topped the "hit list", but it was not an isolated case. Second place went to an IDoc with 170000 status records. And in total there were more than 200 IDocs, each with more than 10,000 status records. What a waste of processing time and database space!

It is clear, that processing of these old IDocs does not make sense. Ok, so what is the reason for this waste of space, time and money?

In a nutshell, I would say it is just a lack of care. The IDocs seem to me like unloved dirty children within the data categories. Unlike master data for example, where a lot of value and effort is put into correct data maintenance, there is no such awareness with IDocs. So, what went wrong here?

1.: It seems there is no adequate monitoring for the processing of IDocs.

IDocs are a bit of a no-man's land. Since they are used in interfaces, their actual nature is technical. But they mostly contain application data. So who should take care of them - IT support or the user department? If in doubt, then... often no one does.

2.: For the same reason, unfortunately, housekeeping often does not take place.
From my point of view housekeeping for IDocs should contain at least these activities:

Set old/obsolete Inbound IDocs to status '68' for avoiding further processing attempts and enabling archiving for these IDocs.
The 'quality' of an Idoc deteriorates with age. Should an IDoc that was created 2 weeks ago actually still be processed? Or is its content now outdated? Has the IDoc been replaced by a newer one so that its processing would cause a double booking?
Archiving!
"IDoc" stands for "Intermediate Document". It is 'only' a container for data to be processed. Once the data is processed, the IDoc usually is not important anymore and can be archived (and deleted in the system afterwards). For this it is important that an IDoc has an appropriate status that allows archiving (according to table STACUST). For most of the Inbound IDocs this should be status '53' (Application document posted), for faulty IDocs status '68'. Notice: status '68' will not be set automatically by the SAP system.

But with only few activities you can achieve a really big improvement for the situation regarding IDocs:

1.: Assign responsibility for IDocs

This could be the most difficult part. The responsibility could take place in the user or in the IT department, depending on the IDoc message type. However, although it might be difficult to find someone who takes over the responsibility, there should be someone for each IDoc type! And the responsibles should be supported with updates about failed and not processed IDocs, e.g. via scheduling RSEBWE02 with suitable selection criteria and mailing the list. Of course a daily check of BD87, WE02 or others is also an option. Anyway, incorrect IDocs should be investigated. It may be that there are missing settings or table entries that prevent processing - this can often be fixed quickly. Or the IDocs contain incorrect data - in this case the sender should be informed and asked to correct the data.

2.: Take old IDocs out of the race

The case mentioned above, in which the system was forced to process an IDoc that was several years old over and over again, is extreme (but true!). Set an age limit after which IDocs are no longer processed. Of course, this limit can also be set in the variants for the SAP standard programs RBDAPP01 and RBDMANI2, but naturally there are many variants that you may not have ongoing control over.

I think 2 - 4 weeks is a reasonable age limit.

Taking old Inbound IDocs out of the race can be done via setting the status to '68' (Error - no further processing). Possibly in your release there is an SAP report RC1_IDOC_SET_STATUS available that can do this, but unfortunately each and every Idoc number has to be specified. So for an automatic solution this report, although it might be available, can not be used.

But it is not necessary to invest time and effort in the development of an ABAP program.

With "Shortcut for SAP systems" it can be done easily by using the "Update table data" function:

First we have to select the connection to the SAP system (1) and the client (2), then we specify the update: the table containing the current IDoc status is "EDIDC" (3), and the field "STATUS" is to be updated with '68' (Error - no further processing) (4).

We restrict the update (5) to the Inbound IDocs (DIRECT = '2') that were created a month ago or earlier (CREDAT LT @( sy-datum - 30 )) and which have a non-archivable status (STATUS in ( select distinct STATUS from STACUST where ARCHFL = ' ' )). Another option would be to select the IDocs with STATUS in ('64','66') ('64' = IDoc ready to be passed to application, '66' = IDoc is waiting for predecessor IDoc (serialization)) - this would be the IDocs selected by SAP standard program for Inbound IDoc processing RBDAPP01 - or STATUS in ('51','64','66') - this would also include the IDocs selected by SAP standard program RBDMANI2 (Repeat processing of IDocs with error status).

Think about it and find an agreement - preferably with the user department or whoever is responsible for the Idocs. A further criteria is the message type, so you can also use different selection criteria depending on the message type.

Once you have finished this, we should think about an automatism - finally we do not want to do this manually again and again.

For this you can either create a variant for the "Update table data" function

or use an XML file:

Personally I prefer using an XML file. With an XML file it is not necessary to create a variant for the "Update table data" function - all necessary information is given in the XML file and it is quite easy to use the XML file also for another system or just extend the XML file for doing the same task in other systems.

Running the "Shortcut for SAP systems" command line tool with the XML file would be done like this:

Here we see a warning ("Sc4SAP command line tool finished with RC 4"). This is because with this execution no IDocs were updated - for today it was already done. But the next days - assuming we turn the execution of the command line tool into automation - IDocs for update will be found and with status '68' there will be no further attempts for processing of IDocs older than 30 days anymore. As the command line tool works on OS level, you can easily add this one-liner into an automatism (simplest way to do this would be the Windows Task Scheduler, but often there are more sophisticated automation tools in use that can do this simple job).

You can also add a list of program RSEIDOC2 in front of the deletion for getting some information about the IDocs to be updated. Just create a variant for RSEIDOC2 with the same selection criteria as you use for the deletion and run the report in front of the deletion. This can be done in an additional task for executing a program in the XML file in front of the update task or via job scheduling in the SAP system.

3.: Implement IDoc archiving

Also this should be done on a regularly basis to avoid the IDoc tables from growing endlessly. Keep in mind that likely the system will be used for system refreshes of your Quality and/or test systems - copying Gigabytes of outdated and archivable Idocs does not make sense, it will cause costs for hardware and it will unnecessarily consume time in the system refresh process.

Find here the SAP documentation about IDoc archiving.

4.: Start now - do it!

Although the 1st task may not be completed yet, the 2nd and 3rd task can be implemented! Every day you postpone this will result in larger Idoc tables and unnecessary space consumption in the database. You will probably never get rid of this! Since there is constant IDoc traffic in the system, reorganizing the IDoc tables resp. the tablespaces to shrink the used space in the database becomes difficult or even impossible.

Once implemented, you will be rewarded with the end of permanent growth in the IDoc tables, which will benefit your SAP system and thus your users and the entire company!

Segregation of duties - setting / adjusting the user group

Shortcut IT — Mon, 08 Jul 2024 14:30:00 GMT

Segregation of duties is important for preventing the company from risks by misuse of authorizations. For user administrators and role assigners in SAP systems the user group - as part of the user master data - in combination with excluding the own user group in the authorizations is a way to implement this.

Read in this article, how simple it can be to ensure that the right user group is assigned to user admins and role assigners.

Even apart from this perhaps special use case, this article shows how easily and quickly "Shortcut for SAP systems" can be used and how the often time-consuming development and deployment of ABAP reports can be reduced!

In a nutshell, this is the principle how to implement this segregation of duties to user admins and role assigners:

1.: Assign a specific user group to your user admins and role assigners, e.g. "SUPER_DLG1" for the 1st, "SUPER_DLG2" for the 2nd person and so on.

2.: Use a specific role for each user admin / role assigner, which has a restricted range for the field CLASS in the authorization(s) for object S_USER_SAS (resp. S_USER_GRP in case you have deactivated the use of S_USER_SAS). Let's name it Z_USER_ADMIN_1 / ...2 and so on.

With these settings the segregation of duties for the displayed user is fine. The user group of the user admin / role assigner is excluded in the assigned role, therefore the self-maintenance is not possible. At the moment. Now. But... the role can be assigned also to another user, and possibly the user group will not be adjusted - this is likely the case when the role is assigned via an IdM tool. Or the user group is going to be changed by mistake. If this pair of role assignment on the one hand and the assigned user group on the other hand do not match, it is possible to maintain the own account. This will be noticed at the next audit at the latest!

So, our requirement is, that each user with a valid assignment of role Z_USER_ADMIN_1 (...2, ...3,...) must be assigned to user group SUPER_DLG1 (...2, ...3, ...) - now, next week, next month - we must be able to rely on it.

How to achieve this? Of course you can develop a program that does this. Here I would like to describe a method faster than this.

In "Shortcut for SAP systems" we have the "Update table data" which can easily be used for this. No development of an ABAP necessary, just spend a minute or two into an SQL UPDATE statement:

Of course we have to choose the connection to the SAP system (1). Then we specify the client (2) - because the function also allows data to be updated in other clients than the one used in the connection. The user group is stored in table USR02 (3) in field CLASS (4), and the new value should be 'SUPER_DLG_1' (5) for all users that (6)

- currently are not assigned to user group 'SUPER_DLG1,

- have user type 'A' or 'S' (means: dialog users)

- are currently assigned to roles Z_USER_ADMIN_1 or Z_AUTH_ASSIGN_1

Now let's make a test. We assign another user group to the user, after this we execute the "Update table data" function.

Clicking on the "View output" button we can see that an update for 1 record was done. And consequently the user group SUPER_DLG1 is assigned to the user.

So this works. But there is one point open: we also want users to be assigned to another group in case the role Z_USER_ADMIN_1 is no longer assigned - finally these users are to taken out of this special user group in case the also special authorizations are not there anymore. For this we have to execute a second update with only 3 small changes:

With this we assign another user group (here: 'IT (Others)') for users currently assigned to group SUPER_DLG1, but without an assignment to the roles Z_USER_ADMIN_1 and Z_AUTH_ASSIGN_1 for the current date. This reverts these users 'back to normal users' and makes them maintainable by the current administrators.

Up to now - in the context of setting up the procedure - we executed the updates of the user group manually. But of course is our goal to automate this.

We can do this with the command line tool with variants...

or by using XML files. An XML file for assigning the user group SUPER_DLG1 (for users with current assignment of the specific roles) and for removing the user group (for users which currently have user group SUPER_DLG1, but are not assigned anymore to the specific roles) would look like this:

With an XML file you do not need to specify single variants for each system you want to apply this procedure. So with respect to use this for more than one system, I personally prefer to use XML files for the command line tool instead of variants. You can easily add more tasks to the XML file - for example for doing the same for the 2nd user admin / authorization assigner or doing this for other SAP systems.

Ok, let's run the command line tool with the XML file.

Ooops, first task ran fine, but the 2nd ended with a warning! Why?

It is just because the 2nd task did not find any data to be updated. "Done. 0 updated rows"

There was no user currently assigned to user group SUPER_DLG1 but without valid assignment to the specific roles. Just nothing to do...

As the command line tool works on OS level, this can be brought easily into an automatism, running daily on your SAP systems. With this you can be sure that the segregation of duties for user admins and authorization assigners is ensured in the system and there are no more findings for it in the next audit.

Extracting data from the SAP system - Part 3 - Output of reports

Shortcut IT — Thu, 14 Dec 2023 13:02:00 GMT

In practice, it is often necessary to extract data from the SAP system, whether for further processing outside the SAP system, for transfer to other SAP systems, or for other purposes. Part 1 of this blog was about extracting table data into a file, part 2 explained the different formats. This part is about how to extract output from reports.

Given is the following use case: as it is for compliance reason necessary to archive information of the users on the system (for a longer time, even if they were deleted), we recognize a gap in the SAP standard functionality. Lots of attributes of the user master data are captured in the change documents, but we can not find any information for the name and the email address. In transaction SUIM we have a report available (RSUSR002) that offers this information. Therefore we decide to schedule the program RSUSR002 with a suitable ALV layout on a daily basis and extract the output into a directory.

Ok, let's prepare some things in the SAP system:

Add an ALV layout (let's name it "/NAME+EMAIL"), containing the user ID, the full name and the email address.
Add a variant for the report (let's name it "4Archiving"), using the ALV layout.

That is all we need to do in the SAP system. Done within a minute.

Now we switch to our application "Shortcut for SAP systems" and create a variant for the function "Execute program" for executing the report RSUSR002 with the variant "4Archiving":

After saving the variant ("PX1/100 user master data for archiving") we make a short test whether the result is as expected. Just click on the Execute button and after few seconds the list is available.

Looks good. The "Save" button in the viewer enables us to save the list in a file.

But of course we do not want to do this manually each and every day. So let us implement a periodic run via the command line tool.

We use the command line tool using the saved variant ("PX1/100 user master data for archiving") and store the output list in a file which contains - beside the information of the system / client - the date of the list. In the file containing the call of the command line tool we specify the filename using the environment variable %date%, then we call the command line tool with

the function as 1st argument,
the variant as 2nd argument
and
the target file for the output as 3rd argument.

Let's test it on the Windows command prompt...

... and have a look into the written file:

Good, works perfectly! Now we can schedule this in a periodic task, using an automation software (this ranges from simple tools such as the Windows Task Scheduler to complex automation applications such as Automic Automation (formerly UC4), ActiveBatch, etc.). In case you use the Windows Task Scheduler, have a look into this blog article which describes doing this.

Part 1: Extracting data from the SAP system - Part 1 - Table data (1)

Part 2: Extracting data from the SAP system - Part 2 - Table data (2)

Extracting data from the SAP system - Part 2 - Table data (2)

Shortcut IT — Tue, 01 Aug 2023 08:25:00 GMT

In practice, it is often necessary to extract data from the SAP system, whether for further processing outside the SAP system, for transfer to other SAP systems, or for other purposes. Part 1 of this blog was about extracting table data into a file with the "View table data" function. There is also another function - "Download / Upload table data" -, offering other possibilities: data can be backed up, restored and transferred to other systems / clients.

In the blog article "How to copy Idocs from one system to another?" it is described how to use this function for copying Idocs from the Production system to a Quality or Test system / client. To be honest, apart from what is written about the "Download / upload table data" function in that article there is not so much to add.

But with regard to the different functions and file types, you might ask yourself what function and what file type is suitable for fulfilling your aim. Therefore, you will find help here to make the right decision. Before we deal with the file types, let us start by having a closer look to the functions.

The "View table data" function (currently) does not offer a selection of records to be processed. This is possible with the "Download / upload table data" function, using the 'Where condition' - a free text field where you can use the ABAP features available in the logical expession in the WHERE clause of a SELECT statement, depending on the release of the targeted SAP system.
Also the files created by the "View table data" function are not suitable for using them as source for an upload in the "Download / upload table data" function. When it is about downloading and uploading the data again (e.g. for a data transfer), you have to use the "Download / upload table data" function.
When your aim is to extract data that is to be used for further processing outside the SAP system (or as input file for processing the data in another SAP system), the "View table data" function should be used.

Both the "View table data" function and the "Download / upload table data" function offer different file types:

The "View table data" function offers a plain text format and a CSV format.
The plain text format is a good choice when the data is to be used for further processing outside the SAP system (or for using an input file for another SAP system) - The record structure corresponds to the table definition in the Data Dictionary.

The CSV format is a good choice for using the file in a Spreadsheet application (like Microsoft Excel). By using this format also header info about the single columns will be written into the file.
The "Download / upload table data" function offers 3 file types:

"ASC" - ASCII format. The table is transferred as text. Conversion exits are performed. The format is easily readable in a text editor. This file type is not suitable when your intention is to use the file for an upload lateron in an SAP system.

"BIN" - Binary format. There is no formatting and no codepage conversion. The data is interpreted row by row; it is not formatted by columns. The readibility in a text editor is poor.
This format is a good choice for saving data and restore it lateron in the same system or in a system with same character set and endianess.

"DAT" - Column-by-column transfer. With this format, the data is transferred as text as with ASC. However, no conversion exits are performed and the columns are separated by tab characters. This format is a good choice for saving data and restoring / transferring it lateron in another SAP system, possibly with another character set and/or endianess. The readibility in a text editor is good, enabling you also to modify the file and adjust values in front of uploading it in an SAP system.

Consider the representation of dates and times, which is dependent on the settings of the user used in the connection! Therefore, when using this format for upload again - maybe in another system - the settings of the user for the decimal notation, the date and the time format need to be the same (transaction SU01, tab "Defaults").

Although the "Download / upload table data" function is suitable for saving / restoring / transferring data between SAP systems: if it is about processing the whole table (all records, resp. for client dependent tables all records in a client), using "Process table data using R3trans" usually is the recommended function.

Part 1: Extracting data from the SAP system - Part 1 - Table data (1)

Part 3: Extracting data from the SAP system - Part 3 - Output of reports

How to copy Idocs from one system to another?

Shortcut IT — Fri, 28 Jul 2023 15:18:00 GMT

Having recent data on quality, test systems and test clients is quite useful for speeding up problem fixing and development. Idocs are a common message type for business data. Read here, how easy it is to provide recent Idocs of the production system to your quality or test system.

In the SAP community you will find lots of questions related to this problem. Usually the answer is to use WE19 for writing the Idoc data into a file. This file then can be used for inserting the Idoc in another system / client. If it is about different systems, typically you have to download the file from the source SAP system and upload it in the target SAP system. The "Inbound File" function is to be processed for each and every Idoc you want to copy.

For a single or a few Idocs, this may be a reasonable task. But not for a larger number of Idocs.

With the "Download / upload table data" function of "Shortcut for SAP systems" copying Idocs can be done very fast and flexibly. By downloading data of 3 tables you will get the data.

EDID4 - This table keeps the data of the Idocs.
EDIDC - This table contains the control records (the header information) of the Idocs.
EDIDS - This table contains the status records for the Idocs (Idoc added, processed, processed with error etc.).

Using the "Download / upload table data" function for downloading Inbound Idocs of message type DESADV since January 2023 would look like this:

After choosing the connection to our target system / client (1), we select the "Download" option and specify an output file ((2) and (3)).

Then (4) we choose an appropriate file type. The file type depends on what we are going to do with the downloaded file. You will find some information in the documentation on the right side of the window (and also here). There are the file types 'ASC', 'BIN' and 'DAT'. As we are going to upload the file in the target system, especially the file types 'DAT' and 'BIN' are of interest. In case source and target system have the same character set and endianess, 'BIN' is a good choice. But a 'BIN' file is poorly readable and should not be modified with an editor. A file in 'DAT' format is processed slower, but does not have these disadvantages. I choosed the 'DAT' file type here.

After choosing the client (5) and specifying the table (6) I reduced the amount of data to my needs: I only want to get Inbound Idocs of message type 'DESADV' created from the beginning of 2023. Notice the 'where condition' (7): the information about message type, direction and creation date is in table EDIDC - by using a subquery I get the Idoc numbers and can apply this selection to the table EDID4 for getting exactly the Idocs I want to copy.

The free text input of the 'where conditon' may look a bit nerdy, but - as you can see here - offers in return a lot of flexibility and possibilities!

After clicking on the "Execute" button (8) the download begins. With the "View output" button I can get some information.

Now let's have a look into the file:

Now we repeat this with the tables EDIDC (control records) and EDIDS (status records). All we have to do is to change the name of the file and the table. The 'where condition' fits also for these tables. Finally we have 3 files downloaded.

Now let's move to the target system / client and do the upload:

We switch the connection to our target system / client (1) and select the "Upload" option (2).

There is a flag 'Overwrite existing entries' which we deactivate (3). If our upload would collide with already existing records (same Idoc number) in the target client, we would receive an error message and the upload would not be done. Typically the Idocs in a Quality system have their origin in the last system refresh from the production system and there is no or less Idoc processing directly on the Quality system. So, if our last system refresh is older than the creation date of the Idocs to be copied (January 2023), there is a high probability that there is no collision. However, deactivating the 'Overwrite existing entries' flag keeps us on the safe side.

Then we choose the file (4) containing the EDID4 records downloaded in the previous step, the right file type (5) and name the table (6). The client will be set automatically according to the chosen connection.

After executing the upload (7) we find that the assumption regarding possible collisions was right, the upload was done without errors. We have a look at the details and see that the records were uploaded.

After that we do the same with the tables EDIDC and EDIDS. And finally we have a look into the Idoc list in the target system / client, where we can see the uploaded Idocs. But there is still a task to do:

We copied the Inbound Idocs from PX1, client 100. Therefore, the receiver information is still addressing PX1, client 100. The header information in the control records (EDIDC) does not fit to our target system / client. We have to update them - at least if we want to work with them (which usually is the reason for the need to get Idocs from the Production system into the Quality system). And, of course, a partner profile in WE20 is required to process them.

Here the "logical system" is used to identify the receiver of the Idoc. Using the logical system is a common method to add an identification to a document - either as receiver or as originator. Also after system refreshes it is an important task to replace the logical system of the source system / client with the logical system of the target system / client. There is a standard transaction to be used for this: BDLS.

In the BDLS transaction we specify the old / obsolete logical system and the new / right one. We have to update our table EDIDC only (you can also specify EDIDS and EDID4 here, but these tables does not contain logical system information). We deactivate the "Start Update of Tables with Special Handling" as it is only about our EDIDC table. After executing the BDLS, the control records look well:

So finally we copied the Idocs from the Production system to our Quality system. We did not have to develop ABAP programs and did not have to struggle with each and every single Idoc in WE19.

For having the possibility to repeat all the previously described steps in a one-liner-command or for implementing some automatism, we can put all the steps into an XML file and use the command line tool.

In the 'xml' folder - supplied with our product - you can find an example (file 'Example_CopyIdocsFromProdToQuality.xml'), where the above described steps are implemented. Of course it is also possible to use Variants (= variants in Shortcut for SAP systems, not those in the SAP system). Personally, especially when it is a process with more than 1 task, I prefer using XML files.

Based on this you can also implement an automatic, daily copy of Idocs. Just replace the '20230101' in the Where condition of the "DownloadTableData" tasks with @SY-DATUM. and schedule the execution with the command line tool daily.

Extracting data from the SAP system - Part 1 - Table data (1)

Shortcut IT — Mon, 24 Jul 2023 14:00:00 GMT

In practice, it is often necessary to extract data from the SAP system, whether for further processing outside the SAP system, for transfer to other SAP systems, or for other purposes. SAP offers several possibilities via the SAPGUI. Probably everyone has already exported a list to one's computer or downloaded a spool request.

This works quite well - more or less - but it is a manual task. If you need this to be done automatically on a periodic basis, in the SAP standard you have to spend additional effort. Typically, you then end up developing one or more ABAP programs, using some function modules and creating the file(s) with OPEN DATASET, TRANSFER and so on. This will satisfy the need to have a file on the SAP system or on a file server, mounted to the SAP system. If you need the file on your (frontend) computer, you are a little bit lost in case you want to have an automatic process.

How you can automate this without developing ABAPs, you can read here.

It is typically about content of tables or of output of ABAP programs. Let's have a look at table content first. Table content can be extracted with the "View table data" function as well as with the "Download / Upload table data" function of "Shortcut for SAP systems". This article describes how to do it using the "View table data" function. The 2nd part is more about using the "Download / Upload table data" function, especially about the available file formats, and in the 3rd part we will deal with the output of programs (lists).

Table content

Nearly every table in the SAP system can be accessed via transaction SE16 (there are few exceptions only, where this is explicitely deactivated). You will receive a list, depending on your settings in SE16 als plain text or as an ALV list or ALV grid.

Using this in background mode (as it is still our approach to automate this) has some hurdles. You can use the ABAP program for scheduling it in a job (the name will be offered in SE16 > System > Status. Usually it starts with '/1BCDWB/DB', followed by the name of the table). But the report might be generated again, with other selection criteria or another selection of fields to be given out in the list. This can be triggered by any user, accessing the same table. Therefore, there is no reliability for getting the data in the same format as it was when the job was set up. If you want to have this for sure, you have to develop a dedicated ABAP program.

Or you use "Shortcut for SAP systems".

There are several functions that can be used for it, and all of them can easily be used for automation.

"View table data"

With the "View table data" function the whole table can be read, each field of the table will be given out in the list.

If you want, you can specify an upper limit of records to be read. A value '0' means, there is no limit, so the whole table will be read.

Using the dialog function you will receive the list in a dialog window:

Using the "Export to file" the list can be exported to your (frontend) computer or an accessible network directory.

There are 2 file types available: CSV (comma separated values) can easily be used in spreadsheet applications like MS Excel.

In case you chose the CSV format, the data will be exported including header information for the fields.

Same file in a spreadsheet application:

When using the other file type - TXT - the file will look like this:

This format is more suitable for further computer processing. The fields and their lengths correspond to the table definition and if the table definition is not changed, the structure of the data remains the same.

Ok, so far we used the GUI application of "Shortcut for SAP systems". Now let's see how we can automate this data extraction. For this we can use the supplied command line tool. It is located in the same directory and is named "Sc4SAPCmd.exe".

Nearly each function available in the GUI application (= each function where this makes sense) can also be processed using the command line tool. There are 2 methods to call the command line tool:

1) Calling sc4sapcmd.exe with function and variant as arguments (variant = variant in Sc4SAP).

When storing the values for the function in a Variant...

naming it "PX1/100 - table AGR_USERS" and wanting to receive the data in a file "PX1 role assignments.csv", the call of the command line tool would be as following:

For getting the data in a TXT file we would use file extension .txt.

This one-liner can be used in a batch file, in an automation software or whatever. The command line tool will be executed on OS level without any further requirements, execute the "View table data" and write the data into the given file.

Let's test this by using the console:

The 2nd option to call the command line tool:

2) Calling sc4sapcmd.exe with an XML file as argument, containing all information of the function to be executed.

This offers more flexibility than using variants. We don't have to specify a variant, the XML file can be created either with a text editor or also by another application (for example a Chatbot can create an XML file for unlocking a user as a result from a chat with him). All information needed for doing the job is in the XML file.

The above used data, put into an XML file, would look as following:

Get the role assignments of PX1/100 in a file

PX1/100, SYS_BC_01

agr_users

E:\SAP\Reports\PX1 Role assignments.csv

Again we have a one-liner that can be used in a batch file, in an automation software or whatever.

After calling the command line tool with the XML file as argument we will receive similar output to the one created by using function and variant:

After the execution the XML file will be enriched with some information regarding the success of the execution and the result:

So, this is a fast solution to extract table data into a CSV or a TXT file from the SAP system to your frontend or a network storage location. You do not have to develop ABAP reports, it can be easily implemented and scheduled on a regular basis.

Part 2: Extracting data from the SAP system - Part 2 - Table data (2)

Part 3: Extracting data from the SAP system - Part 3 - Output of reports

Merging users of the production system and the QA system

Shortcut IT — Fri, 14 Jul 2023 13:30:00 GMT

Recently I found this article in the SAP community:

A legitimate question. After all, a QA system should have the greatest possible similarity to the productive system after a system refresh. Not only for material master data, FI documents, etc., but also for the users. In case there is a big difference in the amount of users on the Production system compared to the QA system, programs and transactions working with user master data and/or authorizations could run fine in the QA environment and bad and slowly in the Production environment. This is not what a Quality system is meant for!

Coming back to the question for having a combination of the users from Production and Quality system: unfortunately the answer from SAP is correct and this is not possible in the SAP standard. Regarding users and their handling in a system refresh (as well as in a client copy) it is always "all or nothing". In the SAP standard you have to decide which users finally you want to be there in the QA system:

the users of the QA system before the system refresh took place
or
the users of the Production system.

In the first case you have to save the users of the QA system before the system refresh and restore them afterwards.

In the second case you have to spend additional effort regarding the system users. System users and their passwords are typically specific to the system (despite there may be an intersection of few system users). If you just use the users as they were copied from the Production system, your interfaces to the QA system will not work anymore (accompanied by many short dumps on the connected systems).

With the "Copy user" function from "Shortcut for SAP systems" you have the possibility to keep the users from the QA system as well as the users from the Production system, and it even can be done with different approaches:

First approach: keep the users from the QA system and copy non-existing users afterwards from the Production system

Typically in the context of a system refresh the users will be saved in front of the system refresh and restored afterwards. In the "PCA" tool (where the above mentioned question is related to) the component "USERS" with all related tables will be exported / imported using R3trans, consequently all users after the system refresh will be the same as before.

With the "Copy user" function you can transfer all missing users from the Production system to the Quality system:

The "Copy user" function can copy users from an SAP client to another one (this can be a client on the same system as well as on another system) as well as from an SAP client into a file and vice versa. Here we copy the users directly from the Production system to the Quality system. By leaving out the flag in the "Overwrite existing users?" field only those users will be copied that are not existing in the target client. Users already existing will not be touched.

There are lots of flags for the individual segments of the user master data and role / profile assignments. Here all segments and role / profile assignments are used. There is a button "more..." near to the "License" flag. By using this you can specify that the users are already charged on the Production system - depending on how you deal with the license costs, this might make it easier and/or save you money.

That's it! It seems almost too easy, doesn't it? But in fact there is nothing more to do. You can do this manually after the system refresh via the GUI application (as shown here) or via the command line tool, if necessary also integrated in existing automations for the system refresh.

Second approach: keep the users coming from the Production system and add missing users, existing on the QA system before the system refresh was done.

For this you did not process the 'USERS' component according to the PCA tool. After the system refresh you have the same users on the QA system as on the Production system. But you can't leave it like that - at least the system users with their passwords (used in interfaces from other systems) must be restored, just as they were before the system refresh. And then there are other users that may need to be restored on the QA system: there may be dedicated / anonymous test users or a project team with extended permissions.

For this you can also use the "Copy user" function, but - as it is about restoring users - you have to consider this in the pre-work of the system refresh. Just copy all users to be restored into a file:

You can also store the full amount of users into a file and reduce the amount of users to be restored afterwards. Here the selection of the users was done in advance. The small button left from the input field for the users is very useful for this!

As mentioned, of course this has to be done before the QA system is overwritten from the Production system. After it is overwritten, just restore the users from the file:

Decide, if existing users are to be overwritten or not. Typically some system users exist on both QA and Production system. If they have the same authorizations it makes no difference if they will be overwritten or not. If they have different authorizations (e.g. for RFC), it can be necessary to overwrite them from the data stored in the file.

After this the system users are back again - but not with the correct password. The "Initial password" field points to this. You can specify an initial password or - if you leave it empty - a password will be generated. In both cases it will not be the right one, that was assigned before the system refresh.

For adjusting the passwords you can use the "Set password" function:

There is no suitable mass function for assigning passwords to users available in the SAP standard. With SU10 you can generate a password for selective users, but this is not what we need. Assigning given individual passwords to users is possible in the SAP standard by maintaining each user individually only.

Here we got the data for our system users out of an Excel file, where we stored the information. The users on the left, the passwords on the right, both line by line. Once inserted, you can also save them in a Variant:

After clicking on "Execute" the passwords will be assigned in seconds:

And again: that's it. The first approach - keeping the users from the QA system and add others afterwards by copying them from the Production system - seems to be easier, but is based on the export and import of the users from the PCA tool. Or, if you do not have the required license for using the PCA tool, you can achieve the same - as described here. However, this is more common than the second approach - taking the users from the Production system and restore / adjust users (mainly system users). But also this is easily possible! Both user copy and password assignment can be performed fully automatically with the command line tool.

Before I forget: Of course, you don't have to wait until the next system refresh to complete your QA system users. Copying the users of course also works completely independent of a system refresh and can be done at any time.

Audit finding: developer keys in productive system

Shortcut IT — Thu, 13 Jul 2023 08:04:00 GMT

It is a standard check done by many auditors: have a look into the table DEVACCESS in a 'Production' system and check whether there are entries. If there are any, this might become a finding - and it does not matter to the auditor whether your system is not changeable according to the settings in SCC4 and SE03.

This can be easily avoided. And if you have already received a finding, you can delete the developer keys quickly and easily.

There is no table maintenance dialog available for table DEVACCESS, so you have to delete existing entries using another method.

For example, you can do this via transaction SE14:

This would satisfy the auditor's requirement. But it is a task to be done manually. You have to do it for each (productive) system, and you can not take it for granted, that the finding will not occur in next year's audit. It is unlikely, but not impossible, that developer keys somehow find their way into your productive system... but with the following method you can be sure that this problem will no longer be a reason for an audit finding.

Typically, this is where the idea comes into play of developing an ABAP program for this purpose, distributing it to all systems and scheduling it as a periodic batch job in all productive systems.

Here I want to show a shorter, faster and consequently cheaper approach. "Shortcut for SAP systems" comes with a 'Delete table data' function that is useful for granting that this issue will never occur in your system. Using the dialog function, solving the problem would look like this:

The entries in table DEVACCESS are to be deleted for all users not beginning with 'EM_'. Here this is for excluding developer keys from an emergency user. Personally, if there are developer keys for the emergency user I would keep them - however, if the auditor also claims about developer keys of emergency user, you can leave this out for deleting all records in the table.

Up to here, pressing the "Execute" button, we have a one-time solution for this problem. But our aim is to do this on a regularly basis, maybe every week, to ensure that possibly created entries will be deleted again on short notice. And we want to do this on every 'Production' system.

"Shortcut for SAP systems" contains a command line tool that we are going to use for this task. It runs outside the GUI application on OS level. This gives us the possibility to execute it periodically via an automation solution. This can be a simple (and free) solution like the Windows Task Scheduler or a more sophisticated (and more expensive) product like Automic Automation or others.

There are 2 options to use the command line tool:

using a Variant (a Variant in "Shortcut for SAP systems")
using an XML file

Personally, I like this XML stuff. It doesn't need any variants and offers maximum flexibility. An XML file for deleting the entries in DEVACCESS would look like this:

In front of scheduling this periodically we call it directly on the console:

In opposite to the first task the next 3 tasks ended with return code 4. This is because in these systems there are no records to be deleted (the ABAP statement DELETE dbtab also sets SY-SUBRC to 4 in that case). It is a warning that there are no records fitting to the criteria (here: the condition with UNAME NOT LIKE 'EM_%'). We can ignore this - obviously we had the issue with the developer keys in the first system only.

Now let's implement this in a periodic task. Here I use the Windows Task Scheduler for this. After clicking on "Create Task..." (1) we enter a name (2) and a description (3). Here I changed the user (4) - I want to have this independent from my personal user, like I prefer to schedule jobs in SAP also with a system user instead of my personal user - and specified that the task has to be executed independently from a login of the user.

After clicking OK (6) we move to the next tab. A trigger (1) is to be created (2) for running the task. Here I specified that the task has to be run weekly (3) starting on next Monday at 06:30 (4).

In the "Actions" tab we enter the call of the command line tool and enter the XML file as argument.

After closing the "Actions" tab we click on OK. And that's it!

The command line tool writes the result of the tasks back into the XML file. By viewing it, you can check whether (and when) it has been executed.

With the implementation of a regular deletion of developer keys you can be sure, that developer keys in next year's auditing will not be an issue anymore. We did not have to develop an ABAP program and distribute it into every system line, followed by a job scheduling. Adding a new system for this task is quite simple via adding some lines in the XML file.

Pre- and post work of an SAP System refresh

Shortcut IT — Mon, 26 Jun 2023 12:06:00 GMT

It is a recurring task for the SAP Basis: the data of the quality and test systems become outdated, are thus less and less useful and must therefore be periodically updated with data from the productive system. For the SAP Basis, this is a high effort, and also under time pressure. After all, the systems should be available again quickly. With "Shortcut for SAP systems", most of the pre- and post-processing can be automated.

The (simplified) sequence in a system refresh is this:

Step 1: Export the data to be kept (mostly system-specific configuration data) on the target system

Step 2: Database copy from source to target system
Step 3: Cleanup of some data in the target system
Step 4: Import data (exported in step 1) back into the target system

In this article we deal with steps 1, 3 and 4. The database copy (step 2) is not considered here.

In many how-to's that you can find in the web, you are advised to create screenshots and then configure the system based on the created screenshots. This is sooooo outdated! It requires a lot of manual work both in preparation and in post-processing, with all the resulting disadvantages, e.g.:

- If something is forgotten during preparation - e.g. a screenshot - it may not be possible to correct it during the post processing.

- Manual activities are prone to errors.

- The processing times for manual activities are longer than for restoring saved data.

- The manual activities may have to be carried out at "inconvenient" working times, which may again increase the susceptibility to errors.

Therefore, the approach with manual activities should be left behind as far as possible and as much of the pre- and post-processing as possible should be done by backup / restore.

In this article I would like to point to the "PCA" tool and how we can use it. "PCA" stands for "Post Copy Automation" and was developed by SAP for supporting system copies / refreshes. However, you need a "SAP Landscape Virtualization Management (LVM), enterprise edition license" (see SAP note 1589175) to use it. If this is given, you may follow the installation guide and use the Task Manager (transaction STC01) for doing the pre work and the post work.

If you do not have the required license, you will see in this article, how information from the PCA tool can be used and the pre- and post work can be done using "Shortcut for SAP systems".

So let's see what the PCA tool can do for us. Hmm... I already wrote that we need a licence for this, so what else can we expect here? With the PCA tool we can get the information of the tables to be backed up (and restored after the system refresh). Even if the license may not be there and the configuration of the PCA tool requires additional effort, we can use it for getting the information about the relevant tables.

There is a report SCTC_LIST_TABLES available in every(?) SAP NetWeaver system, that is useful for showing the relevant tables.

Ok, there are some values to be given as input: a choice between "Refresh" and "Data Cleanup" and an obligatory "Component". Unfortunately there is no F4 list available for the "Component". However, to shorten this: the execution of this report causes some problems. We do not know the "Component" up to now, and even if we would (try "RFC" for example), we would be faced with an error message:

Despite the program is not running, we can still use it for getting the information. A look into the source code shows that the information about the relevant components is hard-coded in function module SCTC_GET_ALL_COMPONENTS and from there the information about the assigned tables is in Include SCTC_SC_INCL_COMPONENTS.

You can use this program instead. You can find it also supplied with our product in the folder 'ABAPs' (file 'ShowPCATables.txt'). Just create it in the customer name space of the development system. Most of the PCA tables are hard-coded in ABAP source codes, some are determined from the TADIR table (R3TR TABL from (also hard-coded) packages).

Let's have a look at the selection screen of the program:

First part with "Refresh" and "Data Cleanup" is similar to the SAP program. What is this "Refresh" and "Data Cleanup" about?

The "Refresh" option belongs to tables that are to be exported in front of the system refresh and to be imported after the system refresh.
The "Data Cleanup" option belongs to tables that are to be emptied after the system refresh. For example, the tables containing data for spool requests, asynchronous updates and background RFC processing are part of the "Data Cleanup".

Ok, for now, let's ignore the other input fields on the selection screen and look at the tables. We execute the program with the "Refresh" option - to see the tables that should be backed up before (step 1) and restored after the system refresh (step 4):

You will find in the list all components and their tables, that are handled in the PCA tool: ALE stuff, Batch Input, SAP Office, printer, operation modes etc. - lots of stuff that possibly you are already aware of and dealed with it more or less cumbersome by taking screenshots before and repeating the configuration tasks after the system refresh. And maybe there is some stuff you were not aware of in the past.

In addition to the original SAP program SCTC_LIST_TABLES you will also find some information about the table size, the amount of rows, whether the table is client dependent or not and a button enabling you to jump to the Data Dictionary information for the tables (SE11).

Ok, so now we have the information about the components and their tables which are worth to be managed in the context of a system refresh. How can we use this information now?

Have a look at the "XML file" button at the top of the list. Such a small button, but it can save us soooo much work.

The generated XML file is for using it with the "Shortcut for SAP systems" command line tool. The command line tool enables you to execute nearly all functions of "Shortcut for SAP systems" outside of the GUI application, executable on OS level.

Let's try it. We are asked about a location for the file:

After saving the file, a popup with some information comes up:

There is a difference of 'Total tables' to 'Distinct tables'. This is because some tables are listed in more than one component (e.g. the components ARCHIVE, ARCHIVE_ADK and ARCHIVE_CUST have some intersections regarding their tables, same is for components CRM, CRMINERP, PI_BASIS_CRM and some more).

Let's have a look into the XML file. There are 2 of them: one file was created for exporting the data - to be done before the system refresh takes place - and one file for the import - to be done after the system refresh, to recover the saved data.

And what's inside? In the "Export" file we see lots of tasks, one for each component and all their tables from the list:

So this is the export of the data, and accordingly in the "Import" file we see the corresponding import of the data.

For exporting/importing "R3trans" is used. "R3trans" is an SAP utility and is mainly used in the Transport Management System (TMS) environment. R3trans can be used to read, save and import table data from the SAP system - and even independently of the database. Also the PCA tool uses R3trans.

In the file we find some information from the selection screen, e.g. the connection data ("+++ Enter the connection here +++"). So let's go one step back and have a more detailed look on the selection screen.

In the first elements of the selection screen you find the differentiation between "Refresh" and "Cleanup", already mentioned earlier in this article. There is also a flag "Without components NRIV and SNRO". For these components all records of the NRIV table would be considered.

Our aim is to get recent data of the productive system via system refresh - including all the business documents, where lots of them have a document number, received from the number range method based on the NRIV table. If we export the whole NRIV table - the last given number for each number range object is part of it - and restore it after the system refresh, we will run into problems when it is about creating new documents. Assuming that on our Quality system currently there are less documents than on the Productive system, the last given number in the NRIV table is already occupied.

In a nutshell: leave the flag set and with this leave out the components SNRO and NRIV.

For the XML files the area below is of interest: you can create XML files for backing up system specific data or for using them in the context of a system refresh (2), which we do here. In the next input field (3) we enter the connection to our target system. In field (4) we specify a directory for the data files (preferably a dedicated directory for this purpose). This directory has to exist resp. created and of course it has to be accessible from the SAP system. As it is here about a system refresh (where usually "only" the database files are copied from the source to the target system), it seems to be a good idea to use a directory directly on the SAP server.

The fields below deal with performance aspects of the export / import. In this article you will find more information.

Now let's execute the program and generate the XML files again.

Again we have an XML file for export and another one for import. Both now they contain the given connection. Of course we could have achieved the same result by just editing the line with the tag in the XML files - it is "only" XML and can be modified with a text editor.

For our system refresh we miss the "Cleanup" part up to now, so we go again one step back and generate the XML file for the cleanup.

Again clicking on the "XML file" button for generating the files:

The amount of components and tables is smaller than for export / import.

So now we have these 3 XML files:

QX1_SystemRefresh_Export.xml
QX1_SystemRefresh_Remove.xml
QX1_SystemRefresh_Import.xml

I move the files to a dedicated directory "E:\SAP\scripts". Now let's do the export of the data and call the command line tool for processing the XML file for export.

During the execution of the tasks you can have a look into the specified target directory and see the files being created:

After few minutes (which is of course depending on the amount of data in your system) the execution finished.

The XML file is now enriched by detailed information for each task. And at the end of the XML file we find a summary:

Lots of tasks finished with returncode 4 - a warning. Having again a look into AL11 we find for every R3trans file also a log file. Let's have a look into one of them:

R3trans works in general with a "control file" (containing the R3trans command, the specification of the client etc.) and - optionally - with a "command file", which is in fact a transport request. The tables to be processed can be specified in the control file and/or in the command file. For enabling R3trans to use parallelization (performance often becomes an issue for system refreshes), the tables are to be specified in a command file. The warning here results out of the use of a command file. When using the PCA tool, you will find the same warning. Not for every component a command file is used, therefore some tasks ended with RC 0.

Ok, it worked. With this we have done the export of the data in front of the system refresh.

Now let's make a jump to the time when the systems database is overwritten with the one from the source system. As you know some data has to be adjusted - lots of settings like SAP Connect, the server groups, CCMS configuration, TMS configuration, logical paths and files, the SAP license, logon groups, operation modes etc. etc. etc.

We are going to do this by restoring the saved data. All of the mentioned topics are covered in the tables we saved in front of the system refresh - so by restoring these data we will have the same status in the overwritten system as before.

At this point in time you are faced with this situation: your refreshed SAP system currently has the wrong RFC destinations and an invalid license. Both are those from the source system and does not fit to your refreshed SAP system.

Without a valid license operating in the SAP system will be rejected.
And after having a valid license again, possibly jobs and queue processing will start immediately, using RFC destinations taken from a productive system!

The solution for this:

start only the database, not the SAP system
import the license via R3trans on OS level
import the RFC destinations via R3trans on OS level

After that, you can start the SAP system again.

The files 'RestoreLicense.cmd' and 'RestoreRFC.cmd' contain the calls of R3trans for restoring the data already exported in the export phase. You can find them supplied with our product in the 'xml' folder. Adjust the paths and store them either in the directory used for the R3trans files or another suitable directory.

After the database of the target SAP system has been overwritten with the one from the source SAP system, we start with the "Cleanup" task. We already have our XML file for the command line tool, so it is just about executing the command line tool with the file as argument.

Again there are tasks ended with return code 4 (a warning). The log files show the reason of it:

Here a warning was given because all tables were already empty. R3trans had nothing to do, so it quitted with a warning.

There were some more return codes 4, and all of them had the same reason.

If you have a look into the SAP system, you will find some impacts of the data cleanup. For example, in STMS you will see the message "TMS not configured for this SAP System", the logon groups in SMLG and RZ12 are gone, SM13 and SM35 are empty, same with ST22.

The Data cleanup part took ~1 minute only on the system. As it is about deletion of data and the amount of tables is smaller than for the previously done 'Export' part, also the needed time is less.

Now let's do the last step (and the most exciting of all): importing the saved data again. As we already have our XML file, it is again just about executing the command line tool.

Again there are tasks with RC 4, so have a look into the log files.

"source system == target system." Yes, that is exactly what we did - export and import on the same system. We can ignore these warnings, they are the same for every task.

The import tasks needed significantly more time than the export tasks: ~27 minutes on my small system. This is because updating the tables in general needs more time than just reading the data. Beside others indexes are to be considered, rollback segments are to be managed by the database etc.

The log files look a little bit different from the 'export' ones, pointing to the fact that it is here about update / insertion instead of simple reading:

Let's have a look into the system now. The TMS looks fine again, also the batch input sessions and the SAP Connect settings in transaction SCOT look like before the system refresh.

All the configuration settings are the same as before the system refresh. This is a huuuuge gain of saved time in comparison to do the pre and post processing manually, using screenshots and do lots of configuration stuff to get the status of the system back as it was before the system refresh. And, if you up to now used a documentation with lots of manual activities, I guess dealing with system specific data using the above described method is much more complete than doing this using your documentation, isn't it?

Once you have finished the adjustment of the XML files for pre-work (Export) and post-work (Data cleanup and Import), you can use it easily to do the pre- and post-processing as often as you like - so, if there is the demand to do the system refresh every month or even every week, there is no need to be afraid of the previously huge amount of work anymore! At the end it is about 3 calls of the command line tool.

As this is working on OS level, you can also easily schedule these tasks or integrate it in already existing automation solutions. Have your weekends been messed up for this in the past? That can now come to an end.

Finally I would like to give some hints on this topic:

Of course it is on you to decide which components and tables are to be considered. If you do not want to deal with the full scope of data, you can filter the list of the program Z_SHOW_PCA_TABLES and / or de-select tables. Possibly you want to get the batch input sessions from the source system, then just leave out the BATCH_INPUT component for Export, Data Cleanup and Import. Just take the list of the program, think about it, discuss and agree with your colleagues and/or the user department, which components are to be considered and which not.
Filter and selection in the list of Z_SHOW_PCA_TABLES will be considered when generating the XML file - means, components and tables that are filtered out or de-selected will not be considered in the generation of the XML file. Certainly, you can also modify the XML file in a text editor and take out some tasks or add others.
In the 'xml' folder of "Shortcut for SAP systems" you will find XML examples for pre- and post work of a system refresh. Have a look into it, they show some additions: in the pre-work locking and kicking out of users is integrated as well as suspending batch jobs in front of data saving. And in addition to the tables shown by the PCA tool, also the ALV layouts are considered. Consequently this is also implemented in the post-processing example.
So you can inform the users about the upcoming system refresh and schedule the command line tool with the pre-processing XML file e.g. for Friday evening - no need to do it on your own and stay in the office. Turn this into an automatism.
You can significantly speed up the process (especially the import, which consumes the most time) by using parallelization. Find detailed information how to do so in this article.
By default also the users are considered in the PCA tables. If you do not want this, leave out the USERS component. For some reasons it makes sense to receive all users and their role assignments from the source system, just like you receive material master data, FI documents etc. from the source system - finally, it is the aim of a system refresh to get data close to the productive environment.
However, the system users have to be the same as before the system refresh, otherwise interfaces would not work anymore. And possibly project teams etc. are also to be kept.
You also have the possibility to keep all users in the target system (by exporting / importing the USERS component) and copy additional users from the source system to the target system with the "Copy user" function.
Have a look at these blog articles regarding this topic:
https://www.shortcut-it.com/blog/index.php?copying-users
https://www.shortcut-it.com/blog/index.php?merging-users-of-the-production-system-and-the-qa-system
https://www.shortcut-it.com/blog/index.php?how-to-assign-new-passwords-to-lots-of-users
Copying users and setting passwords can also be done in the XML files, so you no longer have to do this manually either.

Copying data from one system to another

Shortcut IT — Thu, 08 Jun 2023 09:00:00 GMT

Sometimes it is useful to copy a table or a bunch of tables from one system to another. For example, to update data in the Quality system, in a test client on the development system or in a sandbox with data from the Production system.

Doing this would either be a task to be done on database level or - to keep moving on the SAP level - you will be referred to the possibility with a "Transport of copies" and use the Transport Management System. For one-time use, this may be sufficient. Doing this with a "Transport of copies" is a task to be done manually, it can not be automated.

If the copy is to take place on a regular basis, possibly daily, the procedure with a "Transport of copies" is out of the picture. Do not expect anyone to be willing to bother with this.

Among others "Shortcut for SAP systems" (Sc4SAP) offers also for this problem a solution. In this article you will find a description how easy it can be to update some data on your Quality / sandbox / training system with data from the Production system - fully automated and on a regular basis.

We can use the "Process table data using R3trans" function in Shortcut for SAP systems (Sc4SAP). In this example we are going to distribute the contents of an own-developed table 'ZCUST_BLACKLIST' from the Production system PX1 to the Quality system QX1. The advantage is better possibilities in solving problems, troubleshooting, debugging, testing, etc.

Doing it one-time, by using the GUI, it is a quite clear thing:

1.: Export the data into a file

2.: Clear the table in QX1

3.: Import the data into QX1

We received a warning, so let's have a look what was the reason for it.

Ok, let's have a look into the log file:

This warning was given as we specified the table to be imported. R3trans can do an import also without any table specification, in that case all data of all tables in the R3trans file will be imported - and then no warning will be given. However, we can ignore it, the result in the database is the same.

I used a file on the frontend in this example. Using a server file would be also possible, but this would require that both systems, PX1 and QX1, have access to the directory. This can not be taken as granted, and unless this is not implemented, I am on the safe side using a directory accessible on my frontend. But of course in this case a user with dialog capabilites has to be used in the connection (type 'A' or 'S').

So, after doing these 3 steps - Export from PX1 > Remove in QX1 > Import in QX1 the data of the table in QX1, client 100 is exactly the same as in PX1, client 100. If the requirement is to do this one-time, we have finished it. But in our case the requirement is to do a daily synchronisation of the table from PX1 to QX1, therefore it is about automation - as I already mentioned we can do this fully automated and on a regular basis.

Sc4SAP comes with a command line tool (Sc4SAPCmd.exe), which allows to do nearly every function on OS level with a one-liner. There are 2 methods to use the command line tool:

Calling sc4sapcmd.exe with function and variant as arguments (variant = variant in Sc4SAP)
Assumed we have saved variants for the 3 steps, the calls would look like this:
   sc4sapcmd.exe r3trans "PX1 export ZCUST_BLACKLIST"
   sc4sapcmd.exe r3trans "QX1 remove ZCUST_BLACKLIST"
   sc4sapcmd.exe r3trans "QX1 import ZCUST_BLACKLIST"
Putting these 3 calls in a simple file, let's name it 'Copy Customer blacklist from PX1 to QX1.cmd', and scheduling this file to execute it regularly (e.g. with the Windows Task Scheduler) does the job. A recommendation at this point when using variants with the command line tool: make a note in the variant about this, it might prevent you from changing or deleting if one day you get the urge to clean up the variants.
Calling sc4sapcmd with an XML file as argument, containing all information of the function to be executed
This offers more flexibility than using variants. We don't have to specify a variant, the XML file can be created either with a text editor or also by another application (for example a Chatbot can create an XML file for unlocking a user as a result from a chat with him).

For using XML files with the command line tool you can find some information here. You can also find some examples in the subfolder 'xml', supplied with Sc4SAP. Also this case - copying data of table ZCUST_BLACKLIST from PX1 to QX1 - is covered in an example file ('Example_CopyTableFromProdToQuality.xml').

<Comment>Copy customer blacklist from PX1 to QX1Comment>

<Task>

<Comment>Extract customer blacklist from PX1Comment>

<Connection>PX1/100, SRV_SC4SAPConnection>

<Table>ZCUST_BLACKLISTTable>

<Client>100Client>

<LocalFile>E:\\SAP\\Transfer\\PX1 Customer Blacklist.r3trans.binLocalFile>

Export>

R3trans>

Task>

<Task>

<Comment>Remove customer blacklist in QX1Comment>

<Connection>QX1/100, SRV_SC4SAPConnection>

<Table>ZCUST_BLACKLISTTable>

<Client>100Client>

<LocalLogFile>E:\\SAP\\Transfer\\QX1 Customer Blacklist remove.logLocalLogFile>

Remove>

R3trans>

Task>

<Task>

<Comment>Import customer blacklist from PX1 into QX1Comment>

<Connection>QX1/100, SRV_SC4SAPConnection>

<Table>ZCUST_BLACKLISTTable>

<Client>100Client>

<LocalFile>E:\\SAP\\Transfer\\PX1 Customer Blacklist.r3trans.binLocalFile>

Import>

R3trans>

Task>

ExecutionChain>

If you have seen XML files before (or HTML files, which have a similar structure using this ... notation), it is not so difficult to understand it, isn't it? The root element is the ExecutionChain, in it is at least one Task (here there are three) with their specific tags. For using R3trans we used the task type R3trans tag with its specific tags Export, Remove and Import.

After calling the command line tool with such an XML file (let's name it 'Copy Customer blacklist from PX1 to QX1.xml')

sc4sapcmd.exe "Copy Customer blacklist from PX1 to QX1.xml"

the XML file will be enriched with some information in Result tags, on task level as well as on the level of the whole execution chain - similar to what we can see using the GUI.

Copy customer blacklist from PX1 to QX1

Extract customer blacklist from PX1

PX1/100, SRV_SC4SAP

ZCUST_BLACKLIST

100

E:\\SAP\Transfer\\PX1 Customer Blacklist.r3trans.bin

2023-06-01T11:56:50.444

1st step (preparation) finished.

Now execute R3trans and afterwards download data and log file...

2nd step (execution of R3trans) finished. Have a look into the log file, please.

Remove customer blacklist in QX1

QX1/100, SRV_SC4SAP

ZCUST_BLACKLIST

100

E:\\SAP\Transfer\\QX1 Customer Blacklist remove.log

2023-06-01T11:56:53.227

1st step (preparation) finished.

Now execute R3trans and afterwards download the log file...

2nd step (execution of R3trans) finished. Have a look into the log file, please.

Import customer blacklist from PX1 into QX1

QX1/100, SRV_SC4SAP

ZCUST_BLACKLIST

100

E:\\SAP\\Transfer\\PX1 Customer Blacklist.r3trans.bin

2023-06-01T11:56:53.804

1st step (preparation) finished.

Now upload the data file, afterwards execute R3trans, then download the log file...

R3trans execution done with warning!

See following information

>>> Using R3trans <<<

R3trans location:

/usr/sap/NPL/D00/exe/R3trans

Datafile :

/tmp/080027B7210A1EEE808C5455D6B0FA0A.tmp

Now calling R3trans:

/usr/sap/NPL/D00/exe/R3trans -w /tmp//080027B7210A1EEE808C5454D465DA0A.log /tmp//080027B7210A1EEE808C5454D465DA0A.tmp

R3trans call ended with RC 4

-> see log file for details.

R3trans execution ( Import ) successful!

Downloaded (bytes): 3.148

Uploaded (bytes): 421

2nd step (execution of R3trans) finished. Have a look into the log file, please.

2023-06-01T11:56:53.804

The XML schema is also provided in the 'xml' subfolder of Sc4SAP (file 'Sc4SAPCmd.xsd'). A huge advantage of the possibility using XML files with the command line tool is its flexibility. XML files can be created either with a text editor or by any application. This makes it possible to use Sc4SAP e.g. by a user self service for unlocking the own user or by a Chatbot, creating an XML file as a result of the users input that he/she is locked.

So finally, depending on which method we prefer to use, we have a either a command file with 3 calls of the command line tool (using the variants for Export, Remove, Import) or a single XML file with 3 tasks. Both can be used to execute it periodically. For scheduling it the Windows Task Scheduler can be used or a more complex product for process automation (like Automatic Automation, former UC4).

Scheduling of the table data copy via Windows Task Scheduler - here by using an XML file

Whatever you choose - the method using the variants or the method with the XML file -, in both cases the data can be easily copied from one system to another, fully automated and on a regular basis. Isn't this a nice way to avoid such tedious and time-consuming tasks as banging around with a Transport of copies, day after day?

"We need this information for each system"

Shortcut IT — Tue, 18 Jan 2022 10:00:00 GMT

With an increasing system landscape you might loose the details for each system: is the system still in use, is someone really working on it? Do we really still need this system or could it be closed to save capacity / effort / money?

Recently this question came up at an SAP customer, who has a quite huge SAP system landscape. The amount of test systems raised up to a number of ~300. Wow, that's really a lot! There are quality systems, sandboxes, project specific systems, systems built up as temporary replacements of others etc. etc. Many of them with more than one client (beside 000), so the number of clients affected is therefore much higher, maybe ~500.

Of course the operation of this huge landscape consumes a lot of hardware, time, people, effort... and consequently money. So the simple demand was: let's try to reduce the amount of systems and check whether the systems are still in use. We are sure, that there are some that can be closed!

Easily said, sounds not like a challenging task. Having a look into the last login dates would give the information whether the system is in use. The user information system in SAP (transaction SUIM) offers this information. But doing this for ~500 clients? This makes this supposedly simple task quite time-consuming and expensive. Delegating this task to a single person is not conducive to making that person happy .

Apart from just executing a SUIM report it would cause additional effort to enable the person to do this job - it is unlikely that the user and the authorizations are given in all systems and clients. So, it will become necessary to request a user and suitable authorizations on lots of systems. Possibly the whole task would take a month or more to be completed.

Let's think about another approach: forward the request for information to the system responsibles. Create a small manual what has to be done, determine the system responsibles and ask them to complete the task and return the result. There is a list of system responsibles, so the amount of people to be bothered with it is known. Finally ~60 people would be harassed. Hmm, some are on holiday or ill, others have to ask colleagues or project leaders, some would have questions, ... boah, also this approach would take time and causes a lot of effort.

Perhaps a more central approach after all: can we somehow schedule the background execution of the SUIM report in all systems? There is a job scheduling system in use, there is also a team responsible for this, ...but scheduling the job in ~500 clients and managing the result of the report seems to save neither time nor effort compared to the other solutions.

And how about the program to be scheduled - or executed manually, in case the scheduling option is dropped out?

There is a report in SUIM with a title promising to offer exactly what we need.

Transaction SUIM, report "User by Logon Data and Password Change" (RSUSR200)

Ok, this program offers a list of the users with their last login date. We can reduce the selection to user type "Dialog". However, this program is designed to get the people that have not logged in for a long time. For our purpose it would be better to have a list of users that have recently logged in, for example within the last 90 days, and containing only these users and not the others haven't logged in for a long time. Too bad, this is not possible with this program. Of course we could use the program, but finally we would get a list of all dialog users with their login date. We would have to execute this program for all ~500 clients and then analyze each and every list. Some of the clients have huge amounts of users (more than 40000), so some of the lists would be quite big.

Output of program RSUSR200

To summarize: possible, but not an ideal approach for our purpose. Finally there is no program available working cross-client, and all result lists of all users with their login dates would end in big lists. It would be better to get a result fitting more to our needs. Thus, let's think about an own developed, new program. In fact, our requirement is quite simple and can be met with a small ~60 line ABAP program.

*&---------------------------------------------------------------------*

*& Report z_get_last_login_dates *

*&---------------------------------------------------------------------*

*& Get some information about last login dates of users, cross-client. *

*&---------------------------------------------------------------------*

report z_get_last_login_dates no standard page heading line-size 1023.

parameters: pDays type i default 90.

types: begin of TLastLoginData,

client type mandt,

lastLogin type usr02-trdat,

userIds type string,

end of TLastLoginData.

data: lastLoginDataTab type standard table of TLastLoginData,

lastLoginData type TLastLoginData,

userId type usr02-bname.

start-of-selection.

" Get last login dates of each client (except 000),

" excluding DDIC and myself.

select mandt as Client, max( trdat ) as LastLogin from USR02

client specified

where mandt <> '000' and ustyp = 'A'

and bname <> 'DDIC' and bname <> @sy-uname

group by mandt order by mandt

into table @data(loginDates).

" Get users which have logged in on that date.

loop at loginDates assigning field-symbol().

clear lastLoginData.

lastLoginData-client = -client.

lastLoginData-lastLogin = -lastlogin.

select bname from usr02

client specified

where mandt = @-client

and ustyp = 'A'

and trdat = @-lastLogin

into table @data(userIds).

loop at userIds into userId.

if ( sy-tabix = 1 ).

lastLoginData-userIds = userId.

else.

concatenate lastLoginData-userIds ',' userId

into lastLoginData-userIds.

endif.

endloop.

append lastLoginData to lastLoginDataTab.

endloop.

" Now print the result.

write:/ 'SID', 'Client', 'Last login', 'In use?', 'User Ids'.

data: inUseLimit type syst-datum.

inUseLimit = sy-datum - pDays.

data: inUse type char1.

loop at lastLoginDataTab into lastLoginData.

clear inUse.

if ( lastLoginData-lastLogin >= inUseLimit ).

inUse = 'X'.

endif.

write:/ sy-sysid under 'SID',

lastLoginData-client under 'Client',

lastLoginData-lastLogin under 'Last login',

inUse under 'In use?',

lastLoginData-userIds under 'User Ids'.

endloop.

delete report sy-repid.

It determines the last login dates of all dialog users (except DDIC and the user executing the program) in each client of the system, gets the user ID's that have logged in on that date and check whether the usage was within the last x (default: 90) days. Notice the last line: after execution the program deletes itself.

Ok, works. Due to the fact that the program works cross-client, we can get our information with a single execution of the program in any client of the system. Now the question is how to get this program into any system.

The ~300 systems, that have to be analyzed, are in lots of different transport domains. We can create a transport request for our small program, but again it seems that a lot of people have to be contacted to import the transport request into their systems. Data and cofile are to be copied into the transport directory, the transport request has to be added into the importqueue and finally the import has to be done. Sounds again like a task bringing a lot of effort with it and needs a lot of time.

Tired of reading why all these approaches are slow, costly and overall annoying?

With "Shortcut for SAP systems" you can perform queries across systems, quickly, very flexibly, on short notice and completely on your own, without bothering many people.

Take a look at the function "Insert / modify program".

Function "Insert / modify program"

Having the source code of a program in a text file on your computer, you can insert it and execute it directly after it has been inserted. In combination with the command line tool, you can run a script that goes through all systems and gets a single, final list of required information!

After inserting and immediately executing a program also the output of the executed program is available.

We can see that the system S22 is in use, but there are 2 clients 200 and 300 which are obviously not in use anymore. So, despite we have to keep the system, it is likely that these 2 clients can be deleted, which would save us database space.

The insertion of a program works independently of client settings or the system changeability. We can use it in every system and do not have to take preparations in the SAP systems. What I have to do for this is to create a variant (meaning a variant in "Shortcut for SAP systems", not in the SAP systems) for each system and create a single script, combining the calls of the command line tool for each variant. Having a look at the previous screenshot you will notice that program name and title are not specific for the current request (of getting last login dates of the users). Instead I used a generally applicable name and title. This allows us to use the variants and script for other requirements for cross-system queries.

For example:

Our colleague Laurel Leaving will be moving to one of our competitors. We need to know on which SAP systems he has a user master record and authorizations that we need to delete on short notice. (ABAP program)
We will change one of our central print servers and need to know on which systems there is an RFC connection to it that has to be adjusted. (ABAP program)
We have received a request from the antitrust authorities for information on all business transactions with our supplier Otis. In which systems do we manage this supplier and have business data with him?

For each of these requests, a small ABAP program could provide the desired information. If a suitable SAP standard program already exists, the following procedure can be applied to the "Execute a program" function in the same way.

Let's proceed with the creation of the variants. For each system we save a variant - quickly done by changing the connection, click on the "Save variant..." button, clicking on one of the already created variants (which puts the name and description into the input fields) and just adjusting the SID in the name of the variant.

Now let's move to the command line tool. For executing the function with the command line tool and writing the result into a file, it has to be called like this:

sc4sapcmd insertprogram

Using this command

sc4sapcmd insertprogram -listvariants > c:\temp\insertpgmvariants.txt

we can get a list of all variants, written into file "C:\temp\insertpgmvariants.txt".

With this file in a text editor, we can easily create a single command file that calls the command line tool for all variants.

For each insertion / execution the output will be stored in a separate file. To get one big list with all the results, we add a simple copy command at the end:

Now let's execute the script:

...

Tataaa! That's it - an ad hoc query with the desired information across all systems, in a single list:

Edit: With version >= 1.2.1 of the command line tool there is the possibility to use XML files so that a separate Variant does not have to be created for each system. Just create one Variant only and name it "Cross system query". In the XML file (let's name it "Cross system query.xml") place a task for each system and override the connection and the output file from the Variant:

Cross system query: get users with last login in the last 90 days

in every client in the sandbox systems to find systems that can be

(possibly) deleted.

Cross system query

S22/100, SYS_BC_01

C:\temp\Query S22.txt

Cross system query

S23/100, SYS_BC_01

C:\temp\Query S23.txt

Then call the command line tool with the XML file as argument:

c:\Programs\Sc4SAP\Sc4SAPCmd.exe "C:\Programs\Sc4SAP\scripts\Cross system query.xml"

Offering the whole thing in an Excel file is a piece of cake. I could do it alone, not have to bother many colleagues or delegate the task to someone else (who would then be an enemy for life if he had to do it manually for each system / client).

Do we need an update of the information after 2 weeks? No problem, this would not create any effort at all. Doing the same with using 180 days in the past as the limit for the last login? Also no problem, just replace the '90' by '180' in the text file containing the ABAP program and start the script again. If another question across systems comes up, I can think about another small ABAP program and still use the same script and the same variants. By having the script available at the OS level, I can even schedule it to run regularly. What a saving of capacity, time, ultimately money, and what a gain in speed to achieve the desired result!

Saving system specific tables - part 2

Shortcut IT — Mon, 03 Jan 2022 10:00:00 GMT

In a previous article we described a method to save your system specific data of your SAP system - data, which can not be taken from another system as it is unique for your SAP system, for example RFC connections and a lot of configuration data belonging to ALE, SAP Office, SAP Connect, the Security Audit Log and many more. In case something goes wrong with this data, it could be a hard task or nearly impossible to restore / reconfigure these settings - there is no other system with the same data, so you do not have any possibility to have a look or create a transport request.

Recently I found this in the SAP community forum (https://community.sap.com/t5/enterprise-resource-planning-q-a/sicf-accidently-activated-the-entire-tree/qaq-p/12537899):

A simple wrong click (which might happen every day, no one is safe from this), and now there is a big problem to get back to the previous settings!

The answer to the question - in a nutshell - is: no, there is no way. There are some tools available in the area of the service definition, like exporting the active services and a mass activation tool, but typically there is no export file, and consequently nothing on which you can start the recovery. And the mentioned tools are specific to the service definitions, so you have to know about them and use them, and in this case it would be necessary to do that manually (because the export feature can be used in a dialog session only - no way to schedule it so that it will be executed periodically and automatically).

But with "Shortcut for SAP systems" you can use a generic approach, fitting to the service definitions as well as to many other settings. As written above we described this in another blog article - focused on the dialog function "Process table data using R3trans", dealing with the tables for the RFC connection. If we want to backup all system specific data periodically - including SICF settings as well as many others -, using the command line tool is the preferable solution.

So here is a description of how you can set up a daily backup of system-specific data for being able to restore corrupted settings within a few minutes if necessary.

Ok, but which tables contain the system-specific data? Difficult to figure out... fortunately the SAP system gives us this information! We can use the "PCA tool" (Post Copy Automation). This tool is part of the SAP Landscape Management, enterprise edition software and SAP Landscape Management Cloud software. To use the full scope of its functions, you must either own an SAP Landscape Management, enterprise edition license or an SAP Landscape Management Cloud subscription. If this does not fit to your situation, you can not use it and likely not all software parts of the PCA tool are available in your system. But the good news is, that in every case we can use it to determine the tables which are worth to consider them in a backup.

1st step: Use the program Z_SHOW_PCA_TABLES supplied with our product (folder ABAPs, file 'ShowPCATables.txt', or download it here) to get the complete list. This is the selection screen:

The main usage of the PCA tool is in the context of system refreshes. With the "Refresh" option (1) we will get a complete list of the tables, that the PCA tool would export in front of a system refresh and import it afterwards in the post-processing. In other words, we will get a list of tables containing the system-specific configuration data.

The block "Data for XML file generation" is about the options for generating an XML file to be used with the "Shortcut for SAP systems" command line tool. The command line tool enables you to execute nearly all functions of "Shortcut for SAP systems" outside of the GUI application, executable on OS level. And it is not only about executing a single function - the command line tool allows us to execute multiple tasks with a single call. This is where we want to go to.

BTW, you can also do the complete pre- and post-work in the context of a system refresh. Read more about this here and here.

As we want to create a backup of the system-specific configuration data we set the flag at the corresponding field (2).

In the input field (3) we enter the connection to the SAP system. Input field (4) is for the target directory where the backup data is to be stored. Here we use a dedicated directory for this purpose (which is recommendable).

Now let's execute the program. We will receive a list of components and their related tables:

So this is the overall amount of components and tables that would be handled by the PCA tool (if you have the necessary license) in the context of a system refresh.

Above the table, you will find a button "XML file". With this button the ABAP program can generate an XML file that can be used by "Shortcut for SAP systems" command line tool for backing up the tables in files.

If you want, you can use the ALV filter and/or the selection mark in the 1st row to reduce the amount of components and/or tables. As this article is about backing up system specific data, we simply take all of the components and tables.

2nd step: Click on the "XML file" button

After this you will be asked for directory and filename of the XML file.

After generating and storing the XML file, you will get some information about the content of the generated XML file.

Ok, let's have a look into the generated XML file.

For every component we will find a tag, addressing the "Process table data using R3trans" function in "Shortcut for SAP systems". In the tag we find the connection given in the selection screen and in the tags the path from the selection screen is used.

As written before the XML file is intended to be processed by the command line tool. So, let's execute it.

With this one-liner the content of all PCA components with their tables will be written into files in the SAP system:

It takes a few minutes. At the end you can see in the last message from the command line tool whether errors occurred or everything is fine.

RC 4... a warning. Let's have a look at the reason for this. You can also have a look into the XML file - the command line tool updates it and writes some information into it. At the end you will find a summary, and also each tag was enriched by a tag.

Having a look into the directory in the SAP system you can see lots of files. For each R3trans file you will also find a log file.

The log files were created by R3trans and contain some information about the tables and the amount of processed records. Here we can find the reason for the warning:

Here some information about a transport request comes up. Why?

R3trans works in general with a "control file" (containing the R3trans command, the specification of the client etc.) and - optionally - with a "command file", which is in fact a transport request. The tables to be processed can be specified in the control file and/or in the command file. For enabling R3trans to use parallelization (expecially for system refreshes the performance often becomes an issue), the tables are to be specified in a command file. The warning here results out of the use of a command file. When using the PCA tool for system refreshes, you will find the same warning. Not for every component a command file is used, therefore some tasks ended with RC 0.

More or less that's all! Using the command line tool and the XML file we can schedule a daily backup of this data. This can be done using a lean solution like the Windows Task Scheduler or a full-blown automation solution like UC4©.

In a nutshell, there is very little effort to set it up:

create a backup directory on the SAP server
create the XML file with the supplied ABAP program (this should be done individually for each system line on the development system, because the amount of tables depends on the SAP release and installed software components).
schedule the one-liner (calling the command line tool with the XML file as argument) periodically.

In total: 5 minutes per SAP system? What little effort for such a huuuuge gain in security in your SAP operations.

If we get into the situation that by mistake the whole SICF tree was activated (like the unlucky guy mentioned above), we can easily restore the data in a minute. Just use the "Process table data using R3trans" function for "Import" and use the file containing the data.

After taking over the data from the XML file and clicking on the Execute button the data will be restored in few seconds.

There is a warning given. The reason for this is visible in the log file:

R3trans found that the data file was created on the same system. Yes, that's typical for a backup.

In case the "Buffer synchronization" was not set, due to table buffering it may take few minutes after executing the import (you may have a look into the open synchronizations by using transaction AL12), but finally the SICF tree will be the same as at the time the backup was made!

So, this would be an easy way to get more security in the operability of your SAP systems. In my opinion this should be done for every productively used system. Even if there are doubts that a simple import of the stored data might not be the best approach to fix a recent problem with lost data, the data is there and could also be used to restore it on another system - as an intermediate system where the data can be verified and adjusted if necessary. Once the data is available and usable, the possibilities are correspondingly diverse. Is the user department missing an important variant for a report which can not be recreated easily? Ok, let's make a backup of the variants on a test system, then restore the variants data from the backup of our productive system on the test system, put the variant into a transport request, release the transport request and finally restore the variants on the test system. Likely you will be the hero of the day.

The alternative to this easily done backup of system specific data is a point-in-time recovery of the whole system - if you exactly know the point in time when the data got lost. In practice, this would be a disaster and the very last way out and likely would create more problems than solving it - I never noticed that this really has been done on a productive system. You could also think about a point-in-time recovery on another (test) system, but also this option has some hurdles: you need to have the hardware resources and it is quite time-consuming. Likely it would take days instead of some minutes. The above described backup method can be set up with less effort and provides a tremendous increase in the ability to quickly fix a problem in your SAP systems.

Automation, automation...

Shortcut IT — Thu, 11 Nov 2021 10:00:00 GMT

“I choose a lazy person to do a hard job. Because a lazy person will find an easy way to do it.” (Bill Gates)

With respect to this statement from Bill Gates (and he is for sure not the only one with this opinion), let's see how we can move into this direction - towards to laziness, avoiding many repetitive, boring, time-consuming and annoying tasks.

Automation is one of the key factors to a well-running IT landscape and it accommodates both our need for laziness and safety in operations.

Although many things are possible in SAP, there are still some gaps that make life unnecessarily difficult, especially for administrators. So if you are an SAP administrator and/or interested in making your life easier during system copies or go-lives, here are ways to make some tasks easier.

Think about:

Re-creating the system users after a system copy
Setting passwords for them
Dealing with the system-specific data before / after a system copy

Edit: With version 1.2.1 of the command line tool this article is a little bit outdated, especially regarding the work for a system refresh. Have a look at this blog article:

Pre- and post work of an SAP System refresh

When using XML files you do not need variants anymore.

In the subfolder 'xml' you will also find example files for doing the pre work in front of a system refresh (identically to saving the system specific data) and for doing the post work (cleaning some tables and re-importing the saved data). The also supplied ABAP program (subfolder 'ABAPs', file 'ShowPCATables.txt') makes it so easy to generate XML files for the purpose of backing up system specific data as well as for automatism of pre- and post-processing in the context of a system refresh. Also user locking, unlocking and logging off, copying users and setting passwords can be automated.

Let's start with saving the system users before and re-creating them after the system copy.

We can use the "Copy user" function to store the system users in a file in front of the system copy and to re-create them out of the file after the system copy. To implement this in an automated process we can use the command line tool, which comes with our product.

First we have to specify the system users to be stored in the file. Some users are not necessary to store and re-create them (for instance the DDIC user and the one used for the TMS), but most of the system users are used exclusively on the system and would get lost if we do not save them. Beside the system users there could be some others to be kept, for instance an emergency user.

We save these values in a variant using the button at the upper right of the window:

Now it is on creating another variant for re-creating the users (reading them out of the file and creating them in the system). If the file does not exist yet, we have to execute the copy process now - otherwise we could not create the variant for re-creating them because the function claims that the file does not exist.

Once we executed the "store" part we can create the variant for restoring the users out of the file. Switch the "From" and "To" data and store the settings in a second variant:

Now we have the variants for storing and restoring the system users. Let us move to the command line tool. It is located in the same folder and is named with "Sc4SAPCmd.exe".

Opening the console and executing it offers some information how to use it:

In a nutshell: 1st parameter is the function, 2nd parameter is the variant. Thus, for copying the users into the file it is this:

sc4sapcmd copyuser "QX1/100 syscopy store"

If I would have given a non-existing variant, the command line tool would claim about this. But I did it correctly, and so the copy user function was executed:

Similar to what we have seen using the GUI application. Ok, it worked.

Because I did not set the "Overwrite existing users?" flag, I can also test the 2nd step. I expect that it works in general, but no users are to be overwritten, so finally it does not cause any change in my system:

sc4sapcmd copyuser "QX1/100 syscopy restore"

...

After a system copy the situation will be different and missing users will be re-created.

There is one issue I have to take into account: the user used in the connection has to be valid even after the system copy! If there is one with the same login data that I can use, fine! If not, I have to ensure that the connection to the system used for the re-creation of the users also works after the system copy is done.

After the system users have been re-created, the passwords are to be set. I use the "Set password" function to do so, the fields for the users and the passwords can easily be filled via clipboard. Saving the data in a variant enables me to execute it via the command line tool.

Back again to the console, using the command line tool:

sc4sapcmd setpassword "QX1/100 systemuser"

...

For me as an admin using the command line tool is not easier than using the GUI application.

But, and this is the real benefit, it enables me to offer simple executable scripts to save and restore the users during a system copy process and set the correct passwords. In automation solutions such small one-liners can be integrated in existing automatisms to be executed at any time.

Now let's have a look at another task mentioned above: dealing with system-specific data before and after a system copy. There are lots of them, e.g. the RFC tables, the TMS configuration, the license key and many more. Maybe you have already some scripts or similar to ease your life for some of these, but usually there is still some gap where you have to prepare or finish the system copy by manual tasks.

I would like to draw your attention to the "Process data using R3trans" function, which can make life much easier regarding system copies. I prepared some variants to save system-specific tables in front of a system copy. As in the scope of a system copy only the database is copied and not the whole filesystem of the application server, I used the DIR_HOME directory to store all the data files.

Regarding the granularity I did it similar to the grouping of tables in the "PCA" tool (you can find some information in this blog article). However, you can also use fewer variants with more tables and group them as you like.

Some basic components of the PCA tool are available in all(?) SAP systems with an ABAP stack. You can execute the program SCTC_LIST_TABLES to show tables of a component. However, due to license issues (the use of the PCA tool requires some prerequesites) the program might deny your request. In that case you can try this program instead, showing all components managed by the PCA tool and their related tables in one big list. And here is the output of the program taken from my local SAP system (NW AS ABAP 7.52).

Having all these variants prepared I made a small script file on OS level and stored it in a subdirectory "scripts":

Starting this file on OS level launches the command line tool with function "Process data using R3trans" with the given variants and saves the data of all the tables in files on the application server.

...

In AL11 we can see all the created files, containing the data that is to be kept and restored for the system copy.

Based on the export variants, the variants for importing the data again in the post-work phase can be created quite easily: just take the variants and for each of the variant just change the R3trans action from "Export" to "Import" and store it as "...import..." variant.

Between exporting and importing the data again some tables are preferrably to be emptied. In the PCA tool there is a differentiation between "Refresh" and "Truncate". Whereas the "Refresh" option belongs to the tables to be exported before the system copy and imported again after the system copy, the "Truncate" option belongs to tables that are to be emptied in front of the import. The list of tables is much smaller than by using the "Refresh" option. For example, the T000 table will not be removed - because if this table is emptied, no login into the system would be possible anymore. Also the RFC tables are not listed, due to similar reasons (without some basic RFC connections, e.g. SAPGUI_QUEUE, lots of shortdumps would occur and simple functions using the SAPGUI would abort).

So, let's choose all of the given tables and create variants for removing the contents of these tables.

Then combine all the "Remove / Clientremove" commands in a script as already done for the "Export" variants and do the same for the "Import" commands. In sum you should have 3 single command files, one for "Export", one for "Remove / Clientremove" and one for "Import" (the last 2 files can also be combined into one). With this you can do the pre-work and the post-work of a system copy via one-liners! And to go one step further, the availability of the functions on OS level even makes it possible to integrate the script files into automatic procedures that you might have already in use for system copies.

Isn't this much smarter as doing the same manual administrative stuff like TMS configuration, SAP Connect settings etc. again and again, consuming more time, possibly (rather old-fashioned) based on some screenshots you had to do before the system copy and with the possible error rate that inevitably accompanies manual work? It saves me manual (and thus error-prone) work at potentially inconvenient hours, and on top of that, the time savings are enormous! The command line tool also makes it possible to integrate these tasks into an automated process, thus avoiding the typical break between automated and manual operations at this point in the flow of a system copy. In this way, manpower and also processing time can be saved. And, to come back to the intention mentioned at the beginning of this article, I can be a little lazier again....

How to assign new passwords to lots of users

Shortcut IT — Mon, 01 Nov 2021 14:00:00 GMT

If you have to assign new passwords to lots of users (or just the old ones to lots of system users after a system copy), it seems you are a little bit lost in the scope of possibilities in the SAP standard.

There is a mass function (transaction SU10), but it still does not allow you to

assign the same password to lots of users
assign already given, indiviual passwords to lots of users.

In transaction SU10 the only possibility regarding passwords is to either maintain every user individually (which is nearly the same as you would do it in SU01) or to generate an individual password for each user in the selection.

This leaves us with a lack of functionality for several use cases. Think about these situations:

Your quality or test system will be overwritten with a system copy of the productive system.

This happens quite often - the need for recent data in a test system to offer better possibilities for testing makes it necessary to do this more or less frequently.
With a client copy or export in front of the system copy you can save the users, and after the system copy you can do the client copy again back into the overwritten client of your quality or test system. But you need another system or some space in the filesystem to do this, and - as written in another blog article - it is every time "all or nothing", means you can not have users and authorizations from the productive system mixed with the project team, some test users and the system users of the quality or test system. Not in the SAP standard.

But for test purposes it could make sense to have the regular users and their authorizations from the productive system, especially when there is a need to consider also some authorization tests. But, as written before, the mix is not possible via SAP standard tools.

In any case it is necessary to keep at least the system users in the quality or test system. Usually they are system-specific, the passwords are individually assigned and after the system copy all these users with the correct passwords have to be available again. Otherwise the interfaces with the connected systems will not work anymore. That's why in most cases the strategy is to keep all users of the overwritten system and the above mentioned client copy or export and import tasks have to be done. This takes time and increases the pressure on the admins to work as quickly as possible to offer the system as early as possible.
There are lots of new users in the system which need an initial password.

Maybe the initial password is the same for each of the new user (not the best approach, isn't it?), maybe the initial password(s) is/are already published. This could be the case on a training client. There might be users like STUD_001 with initial password havefun_001, STUD_002 with initial password havefun_002, and so on.

Or these lots of new users should get generated initial passwords. From security point of view this would be the best option. And this use case is covered in the SAP standard. In transaction SU10 you can select lots of users and generate individual passwords for them. You will receive a list of the users and their passwords.

Ok, this is usable, isn't it? But...there is still one thing where our product offers additional value. We will see it later.

With "Shortcut for SAP systems" we offer a flexible and fast possibility to assign the same password or agreed individual or generated passwords to lots of users. Let's have a look on it:

There are 2 tabs: "Same password to users" and "Individual passwords to users". Here you see the finished assignment of the same password to 24 test users. A typical use case is the start of a test phase where the teams were already informed about the test users and their passwords. Doing this with the SAP standard functions would make it necessary to maintain each of the 24 users individually. Possible... yes, but annoying, boring, time consuming.

If it is on assigning individual passwords to lots of users, we have to look at the other tab.

Here you can see the finished assignment of the individual passwords to 36 system users. Without our tool, password assignment would require maintaining each of these system users individually. The passwords for the system users are very cryptic and complicated, due to the use of the password generator feature when they were created (well done, this is how passwords for system users should be given). Entering them manually without errors would be a challenge, so without our product's "Set password" feature, this would be a copy'n paste task via the clipboard - in this case 36 times. Looks really like an interesting task, doesn't it?

Using the "Set password" feature you can do this in a single step, done in few seconds. The text fields for users and their passwords can be filled via the clipboard, and as admins usually have lists of system users and their passwords, the whole task can be done in a minute. This is much easier and much less error-prone.

And, especially in the context of system copies, we can go one step further: by using the command line tool this can also be integrated into automated processes, so you don't have to do it manually. Find some information about our command line tool in this blog article.

There is another button available after the assignment of password(s) is done: "View details". Hmm, what details?

Let's come back to the above mentioned 2nd use case: There are lots of new users in the system which need an initial password.

If this initial password should be a generated one, this is the use case that is covered in the SAP standard. You can select the users in SU10 and assign a generated passwords for them, and after this is done you will get a list with the password information. This list can be used to inform the users individually about their initial password - after you have determined the email addresses of the users. And this is the catch.

You can do the same with the "Set password" function, and, to come back to the "View details" button", you will be rewarded with a list containing all the necessary user information to shorten the task of informing the users.

Other than in the SAP standard you will get the information of the email address of the SAP user. You can use the "Export to file" button to get a CSV file that can be used for a serial email. With "Shortcut for SAP systems" also an ABAP program is supplied, that can be used in any SAP system with a configuration for sending emails to inform the users about their initial passwords (subfolder ABAPs, file PasswordMailer.txt). Just select the CSV file, adjust some parts of the serial email and execute the program - very easy!

But of course also a serial email with email clients and/or Office software is possible. Here you can find some information how to do this

with MS Office (german / english 1, 2)
with LibreOffice (german)
with Mozilla Thunderbird (german)

Once set up and saved, it is easily reusable.

In the end, with the "Set password" feature of our product there are ways to save a lot of time and reduce the probability of errors as well as duration and time pressure.

What a waste of time...

Shortcut IT — Mon, 16 Nov 2020 12:53:00 GMT

Yesterday I saw it again. Indeed, I see it nearly every day or at least every week.

In the ticket system a request raised up for an emergency user. The reason was "... to delete entries in table xyz using SE16."

The ticket was raised yesterday at 8:16 and closed today at 7:55. It took nearly 24 hours to adjust the data, just because there was no maintenance dialog available. One can assume that during this time frame some functions did not work, extra effort was necessary, business processes were out of order or in a delay and had a negative impact on customers side etc. etc...

Most cases of these kind of tickets are related to productive systems, but also some were created for test or quality systems. I don't know the details about lots of the named tables, but of course the experts know them. But they did not find a way to correct the data because in the SAP system there is no function available for maintaining the data of this table. Or maybe there is one, but the system and client settings related to changeability pretend them to use it. This could be an omission from SAP or not. If everything is fine, it is not necessary to have a maintenance possibility for every table. But in reality we don't have the situation that everything is fine and all IT systems run well and error free. In that case lots of us would not have a job.

Avoiding the above mentioned barriers and fixing those issues in an SAP system is still the same since 20 years or more.

I guess some or most of you know the trick: use SE16 to display the records of the table, jump into the details of a single record, switch on debugging, press Enter and after replacing the value of the variable CODE with 'EDIT' or 'DELE' in the debugger you can maintain resp. delete the record. Or you use transaction SE16N, which offers similar possibilities - dependent from the SAP release (it is not available in every SAP system) and some SAP notes dealing with the deactivation or reactivation of the EDIT function.

With the debug+replace authorization (S_DEVELOP with object type 'DEBUG' and activitiy '02) you can switch SE16 and/or SE16N into the 'EDIT' mode, but of course this authorization is typically given very restrictively. The emergency user should have this authorization, that's why all these tickets come up. If you have some kind of "firefighter" tool in use this is an alternative to the emergency user, however, the procedure and also the overhead is similar or the same.

Come on... has there really been no progress in the process of data correction for decades? Even with test and quality systems, are time-consuming ticket workflows necessary for the team to correct some data?

Doesn't this really bother anyone, this waste of time and resources?

With "Shortcut for SAP systems" you can eliminate this burning of time and money. You will get an editor for every table in the system, independently from an existence of a maintenance function, the table definition and the system and client settings. And there is also a solution in case the required data correction overtaxes manual table maintenance due to the amount of data to be corrected.

Let's make a short tour and start with the table editor. You can find this function in the "Table data" section.

After specifying the table name you can press the "Edit the data" button to start the table editor. You can enter any table here - no matter whether the table maintenance is allowed in the settings (SE11) or there was no table maintenance dialog generated or the system or client settings regarding changeability would pretend you from editing the data via SE16 or SM30.

If the table contains lots of records you can select the relevant ones using the "Selection criteria" table and the elements below that table. Just choose a field, specify a value / range / pattern and press the "Add" button to populate the selection criteria table.

In this case this is not necessary, the table used for this demonstration (USRFLDVAL) contains only few records. As the table maintenance uses the SAPGUI a user with dialog capabilities (type A or S) is necessary in the used connection.

After pressing the button a session with the SAPGUI is started and a popup comes up:

There are 2 possibilities regarding the editor: 1) the SAP standard dialog of transaction SE16N with activated editor mode or 2) an own editor, coming with "Shortcut for SAP systems".

SE16N is not available in every system, in that case the own editor is to be used. This is the case in BW, SRM, CRM, SCM systems and also in my local SAP demo system.

With this editor you can flexibly modify the data. Changing the existing records, adding new records or delete some... possibly within a minute, without any debug/replace tricks. It saves a lot of time in comparison to a workflow for requesting an emergency user, involving all the people that are used for this. Disturbances on your systems, which have their reason in few invalid data, can be solved quickly. And what is stated on our website is still true: there is nothing to be installed into your SAP system for enabling you to use this solution! No transport requests, no AddOn's.
Using the "Cross-Client" flag, it is even possible to maintain the data in other clients:

Ok, there might be situations with invalid data in which the maintenance of the data in an editor is not the suitable solution. Think of a necessary update of thousands of records in a table. It can be done with the editor, but there is another function fitting more to the need of updating lots of records. You can find it also in the section "Table data".

Looks a little bit more technical, doesn't it? With this function you can do updates on specific fields of a table, dependent from some selection criteria, specified in a "where" clause. Think of a situation where a small correction program is necessary to adjust the data - but this function does the job for you and you don't have to write a correction program on the DEV system, put it into a transport request, release it, request the import on QA and PROD system, wait for the import and execute it. This saves you effort, but what is even more important in many cases: it saves you time - time in which business processes may not work, your ability to deliver is not given, your customers have to wait!

You can do this using a SAPGUI session (you would get a short information in the SAP system about the amount of updated records) or without - in that case also a system user can be used. Press "Execute" and the update of the data is done within a minute.

If you have to delete lots of records in a table (or maybe all) you will find a "Delete table data" function, which looks and behaves similar to the update function.

But of course the needed correction of data could have a little more complexity and the above mentioned functions are overtaxed. In that case the creation of a separate correction program is necessary. Then other dependencies come up: you need the DEV system, a developer with the necessary authorizations and a developer key, the basis team for importing the transport request.

Don't get me wrong, I don't want to state that the usual build and deploy process for programs is nonsense! But in emergency cases it should be possible to react quicker than the regular build and deploy process allows! It is all about keeping or re-establishing the operability of of your company's IT system. It must be permissible to put the usual formalisms on the back burner and to fix the problem as quickly as possible.

With the "Insert / modify program" function "Shortcut for SAP systems" offers an upload function. You don't need a developer for this, no developer key is necessary and also no changeability settings for repository objects in the system / client have to be adjusted. Just put the source code into a file on your computer and upload it. If you want, you can also execute it immediately after the upload.

To recall the picture shown at the beginning: with "Shortcut for SAP systems" you eliminate the usual barriers that prevent you from quickly correcting data.

Once again: it is not about demonizing the usual processes and questioning their suitability. But if your company comes under considerable pressure or is at risk due to a malfunction of your SAP system, you should be able to react and solve the problem quickly. The satisfaction of your customers and thus the success of your company depends on this. The availability of your processes should be given top priority - clear before following rigid process descriptions. And with "Shortcut for SAP systems" you will get the possibilities.

A short story of a tormented SAP system

Shortcut IT — Thu, 09 Jul 2020 11:26:00 GMT

It is quite normal that a complex IT system like an SAP ERP is occasionally subject to disruptions. But does so much time really always have to pass before the system and its users are freed from the pain?

This is an appeal to remember the real goal.

Recently I was able to observe it again: a small error in a transport order with a big impact. It happened like this: at the customer's site, there is a control table for calling function modules developed in-house in user exits. In this table the function modules are maintained as well as an "active" flag. Before the function module is called, it is checked whether the exit is active. If so, the function module is called.

Now it appears that the change in the control table has surpassed the development of the corresponding function module, and the transport request with the "active" flag arrived in the production system before the function module. It happened as it had to: within a very short time, important transactions in the logistics area terminated with runtime error CALL_FUNCTION_NOT_FOUND. Not only about 800 dialog users, but also several interfaces were affected. The system was busy writing short dumps, and new error messages about this error were piling up in the ticket system.

Within a relatively short time, the cause was clear - the above-mentioned new record in the control table. The malfunction in the production system would now have been very easy to be fixed: just delete the record or remove the "active" flag, and the failures would not have occurred anymore. Well, easier said than done. The control table could not be managed in the system, but is delivered to the production system exclusively via transport requests. So the detour via the development system had to be taken, where the "active" flag was removed and this change was written to a new transport request. The transport request must be accompanied by documentation, and then an "emergency transport" process must be performed to import this transport request into the production system. A standardized e-mail is then sent to the trip for approval. From the beginning of the error, it took 1:45 hours to fix this error in the production system. During this time, the system suffered from almost 10,000 transaction failures, each with a short dump. And for the same period of time the ticket system received more and more fault reports, all related to this error.

In my opinion, the reason of this waste of time is that the repair of the malfunction in the production system had to take an unnecessary detour via the development system. If the "active" flag had been removed directly in the production system as soon as the cause was identified, the duration of the malfunction would have been reduced to about 20 minutes. Of course I don't mean that every malfunction should be fixed directly in the production system. But in this case, where the cause was clear and the elimination of the malfunction (the removal of the "active" flag) did not even have to be checked in a quality or test system, an unnecessary amount of time was lost, during which the system, its users, the interfaces, and ultimately the company and its business processes suffered. It was painful to have to look at this in the light of the fact that immediate troubleshooting would have been actually quite easy. With our product "Shortcut for SAP systems", the "active" flag or the whole record could have been removed within 1 minute.

By the way, the customer values 1 hour downtime of the system at over 200.000€. If we assume that the malfunction could have been fixed within 20 minutes, the delayed troubleshooting after 1:45 hours caused additional costs of about 280.000€. Ok, the system was not really "down", but working with the system was only possible in a very limited range (and not at all in the concerned logistics transactions). So, there still remains a considerable financial loss.

Hence my appeal to the many brave employees in IT support and especially to the superiors responsible for IT support:

The highest goal must be to get the productive system running smoothly again as soon as possible.

Adjustments on the development system can be carried out in peace after the malfunction in the production system has been eliminated - that is only secondary. Equally secondary are process descriptions, documentation, approvals etc. This can also be done after the fault has been resolved.

So, here are my thoughts and my recommendation:

Do not rigidly adhere to processes and approval paths that might delay fast and efficient troubleshooting. Don't blindly trust the voices of auditing companies telling you that every change must be documented, approved and tested before it reaches the productive system. Also, be careful when security experts try to convince you of a "minimum principle" of authorizations for each and every user working in the system - it is likely that at least for the people that have to fix a problem the limited authorizations won't be sufficient in an emergency case.

Both, auditors and security experts are not your customers! Your customers are those who can expect you to deliver the goods on time. These customers should be the focus of the troubleshooting - not the consultants whose customer you yourself are.

Process descriptions describe the normal, everyday situation. In an emergency situation acting purposefully and rationally should be allowed to beat any process description! Do not allow rigid process descriptions to paralyze you resp. your company and leave no opportunity for appropriate, reasonable action.

Set up a team to troubleshoot your production systems and provide them with the necessary authorizations, tools and, most importantly, the necessary confidence.

This can be a really small team. And these people don't have to be module experts, but employees who have a good knowledge of the SAP environment and who can act as an extended arm of the module experts in the systems if necessary. Think of this team as part or as an extension of your regular support team.

It is not even necessary to recruit new staff. The members of this team may well have other full-time tasks, it is not necessary to reserve the working time of the members of this team exclusively for possible emergencies. But if your system - and therefore your company! - gets into a precarious situation, it is important that at least one member of this team is available and can act without first having to overcome bureaucratic or formalistic hurdles.

There are some software solutions on the market for dealing with emergency users for situations like these, for example "SAP Access Control" (as part of the "GRC"), or "ISPICIO Emergency Access Management für SAP". I do not know them in detail, but you should refrain from approval steps for the above mentioned nominated team members! If, for compliance and security reasons, you create a process that includes approval steps before your team can save your company, you could end up losing valuable time again!

Give these employees the confidence they need - don't let them, their commitment and motivation fail and despair because of missing authorizations and missing tools.

It pays off - by the satisfaction of your customers with your company's reliability.

Saving system specific tables using R3trans

Shortcut IT — Thu, 19 Mar 2020 16:00:00 GMT

Here is a short description how to back up system specific tables with Sc4SAP (and how to restore them in case you need it).

We take the RFC connections for this example. In case they get lost you are in big trouble. Outgoing data can not be sent any longer, functions using RFC connections are not working any longer. Maybe in a very short time your system produces thousands of short dumps.

You don't have the possibility to get back this data from another system as every system has its own individual RFC connections. There is no way to import the data via transport request or similar. Without having a backup of the specific tables the only chance would be to set up a new system via system copy using a full backup of the concerned system. A huge effort, and very time consuming - time during which your SAP system is not working properly!

BTW, indeed we noticed complaints that during an SAP Upgrade the RFC connections got lost :-( Thus, in front of an SAP upgrade saving the RFC connections might be a good idea.

The procedure described here might also be a solution when you have to do a system copy / refresh (e.g. from a productive system to a quality or test system to get recent data on them) and there is no "PCA tool" (see below). Just export the RFC connections of the quality or test system in front of the system copy and restore them afterwards.

Edit: Meanwhile, using our command line tool and an ABAP program supplied with our product, saving system specific data becomes very easy and can be implemented within minutes. Have a look into another blog article: "Saving system specific tables - part 2".

And it can also be used for doing the pre- and post-processing in the context of a system refresh: Have a look into this and this blog article.

Ok, let's start:

1.: Determine the tables that are relevant. In general it is not really easy to get the knowledge about the related tables of a specific area, but here - for system specific tables - we have some help from the "PCA tool" (Post Copy Automation). Have a look into the function module SCTC_GET_WHITELIST. This function module gives information about the components managed by the PCA tool and their related tables, all listed in the source code of the function module.

BTW, you might get more ideas related to system copies and saving of system specific tables by viewing the components and their related tables in this function module. Beside RFC connection data there are also other suitable tables that are worth saving, especially with regard to the preparation and post-processing of a system copy: ALE configuration, the SAP license, ECATT, logical files / paths, TMS configuration, variants, web services etc. etc. etc...
You can also execute program SCTC_LIST_TABLES to show tables of a component. However, due to license issues (the use of the PCA tool requires some prerequesites) the program might deny your request. In that case you can try this program instead, showing all components managed by the PCA tool and their related tables in one big list. It is also supplied with our product, you can find it in the folder 'ABAPs' in file ''ShowPCATables.txt'. And here is the output of the program taken from my local SAP system (NW AS ABAP 7.52).

2.: Use the "Process table data using R3trans" function to create a data file with all contents of all tables related to RFC connections. Some of the given tables are client-dependent (other than most of the RFC tables), therefore you have to specify a client. Sc4SAP allows us to specify '*' in the client field, which means all clients. Of course you can also specify a specific client. Regarding the RFC tables saving the data for all clients is to be recommended (see footnote #1).
Depending on the release of your SAP systems maybe not all of them exist in your system. Sc4SAP checks this and indicates missing tables. Just leave them out.

As our purpose in this example is to create a backup of table contents we place the backup file on the application server. Of course you can also use the frontend for storing the data. In that case you have to use an SAP connection using a user with dialog capabilities because of the download of the data file to your computer. In this example storing the data on the application server seems to be a good idea, and DIR_HOME seems to be a suitable directory, but you can use any directory that is accessible by the server. If the user used in the connection is a system user, you have to remove the flag in the "With SAPGUI" field.
Now let's start the export. As there is (usually) a small amount of data in the RFC tables it takes a few seconds only and the Messages / log area indicates finishing the task:

3.: Looking into the log file should definitely be done, especially when you do this a first time. If we would have used the Frontend as the location of the data file, this could be done by a mouse click only, but in this case we used the server. So we use transaction AL11 for viewing the log file. Selecting DIR_HOME we can see the 2 files created:

The log file contains the information about the written data:

That's all for the moment. It took only some minutes to create a backup file of the RFC connections - and hopefully we don't need it, but in case the RFC connections would get lost (shit happens!) we can use this backup file to create them again. When there is a change regarding the RFC connections of course also the backup file should be refreshed. Which takes me to another nice thing: the command line tool, that comes with "Shortcut for SAP systems". You can find some information in this blog article. Using it makes it possible to schedule a periodic backup of system specific data - so you don't have to do it manually and it also works at weekends and if you are on holiday. You can put a one-liner for every system in a command file on OS level and execute it in the shell or use it in an automation software (this can be a lean solution like the Windows Task Scheduler or a full-blown automation solution like UC4©).

Use the command line tool to schedule a daily backup of system specific tables!

Let's assume we get into the trouble of lost RFC connection data and want to restore the RFC connections. Or we made the backup in advance of a system copy and want to get back the original data in our refreshed system.

Use the "Process table data using R3trans" again for restoring the data. In case the current RFC data in your system is corrupt you should use the "Remove / Clientremove" option first. Specify the tables to be deleted and it will be done. You don't have to use all tables that were written into the data file. In case you have to restore only few or a single table you can specify them / it in the text field and data of other tables will not be processed.

In our example we don't know if only one or few tables are concerned, therefore we take all RFC tables and import the data back into the system:

In DIR_HOME we now see also the log file for the import:

The data is imported, the RFC connections are restored and work properly. The operability of our system is given again, and it took only few minutes!

If you are interested in covering more system specific tables and / or how to execute a backup automatically and periodically, you can find more information in this article: Saving system specific tables - part 2.

Footnote #1: There is one special table here: RSECTAB ("Secure Storage: Memory for Encrypted Data"). There is a special issue regarding the "Secure storage": its data is stored using a special value '___' for the client.

In the R3trans log file shown above we can see 12 entries of that table exported and also imported lateron. It is not possible to view the entries using SE16 or similar tools (SE16 shows a message "You can not display RSECTAB with the standard tools"). But with transaction SECSTORE you can see the records:

These are the 12 entries mentioned in the R3trans log files, and you can see that nearly all of these entries have this strange value '___' in the client field. They contain passwords (in an encrypted form) for users used in RFC connections as well as in some other areas (e.g. for the database connection - at the left side in transaction SECSTORE you see some applications using the secure storage), and the intention for using this value is to achieve more security regarding sensitive password information in an "invisible" client. Well, for the regular users SAP did it not like this and stored the hashed values for the passwords beside all other client related data in the same client. I guess the reason for implementing the "Secure storage" with this "invisible" client is that the passwords are encrypted instead of hashed like the usual passwords, and of course the SAP system needs to decrypt the value back to the password for using the credentials in the RFC connection. This offers a wider attack surface for hackers than a hashed value (which offers no approach to get back to the original value except time consuming brute-force attacks), and this probably explains this additional hurdle.

BTW, if you are interested in SAP's password security, this blog article might also be worthwhile for you.

But this is really an extraordinary and special case - you don't have to expect similar stumbling blocks with other tables than those related to the "Secure storage". But for the RFC connections this table likely has to be considered, and that is the reason why we specified client '*' when we exported the RFC data. Otherwise we would not catch the entries with client '___' and there would be no possibility to restore also the passwords of the users in the RFC connections in case these entries would get lost. SAP note 1532825 describes some trouble with the Secure storage after a system refresh and offers some workarounds - but it seems to me that the better solution would be to avoid the hassle and effort of repairing invalid data by making a backup of that data and restoring it after the system copy...

Oh no.. missing authorizations

Shortcut IT — Thu, 16 Jan 2020 14:10:00 GMT

This article deals with a usually unpopular topic: authorizations.

A lack of authorization is usually an unbridgeable hurdle, so that it is not possible to make any progress with the completion of our task. Maybe a colleague has the missing authorization and can help out, but it usually boils down to the fact that you have to apply for the missing authorization somehow - after identifying a suitable role - and then (hopefully) get it after a while.

This might be ok if there is some time to complete our task. If it can be done also next day or week, ok, let's wait for the approval.

But it may also be necessary to obtain the missing authorization as soon as possible, for example

during a go-live: usually very tightly planned, usually carried out on a weekend and with no reserves in the schedule. The approvers may not be available at the weekend. The entire schedule and with it the timely availability of the system is at risk!
in normal productive operation. For example, an imported transport request may cause a further authorization check to be processed. A missing authorization for a system user immediately brings an interface to a standstill. The system may be flooded with thousands of short dumps. In such a situation, there is usually no time for time-consuming authorization requests or role changes in the development system and subsequent transport into the quality and production system. Hey, your business is at harm!

"Shortcut for SAP systems" (Sc4SAP) offers some possibilities to help you.

1.: Assign a role and/or a profile

This could be suitable approach in case you know which role or profile is missing. Simply assign the role or profile to the user and the authorization is given.

This function needs a user with the appropriate authorizations for role assignment (S_USER_AGR, S_USER_GRP etc.). The impact and the routines behind are the same as used in transactions SU01 or PFCG, including change documents for the assignment. However, an authorization allocator is not needed on-site, because in Sc4SaP you can also use a system user. Thus, this is an appropriate method to assign missing authorizations in case there is no authorization allocator and/or there is no way or time to follow the regular approval process for authorizations (e.g. a go-live at the weekend).

2.: Assign a "hidden" authorization

This could be suitable approach in case there is no role or profile available for the needed authorization or the role or profile is too extensive or would cause an overflow (see SAP note 410993). Because this function can be used for addressing other clients it might also be a solution in case you don't have access to a user in the targeted client with the necessary authorizations for the 1st function (see paragraph above).

There are 3 options to assign the missing authorization:

1.: you can assign a full authorization for a given authorization object.

2.: you can assign the authorizations of a role.

3.: you can assign the authorizations of a profile.

For the user used in the connection it is not necessary to have the authorizations for role / profile assignment. The method behind it is completely different from the regular role / profile assignment. In case your SAP system uses more than 1 application server there could be a latency for the given authorization to be effective (usually up to 2 minutes, depending on the value of profile parameter rdisp/bufreftime).

This method has some peculiarities that you should be aware of:

There will be no change documents for the assignment. So if you want everything to be correct, you should document it accordingly and have it approved (retrospectively) if necessary. Hint: all actions in Sc4SAP are logged in the "logs" directory, so you can catch up this afterwards when there is time to do so.
The user will not be listed in the SUIM reports when you search for users with the assigned authorization!
The authorizations assigned this way will disappear at the next time the user buffer for the user will be refreshed. This could be caused for example by a "normal" role assignment for the user, a user comparison run or the first login of the user, and there might be some more possible reasons.

3.: Assign a reference user

This is an appropriate method to fix authorization problems in the system without spending time to find the concrete missing authorizations. Of course also an assignment of profile SAP_ALL would be a possibility, but maybe there are some reasons for avoiding this (for example, a strict ban on the assignment of SAP_ALL, impending costly compliance tasks, etc.). In that case using a "reference user" might be a solution. In a nutshell: if there is a user A and for this user a user B is maintained as a reference user this will have the impact, that user A has all authorizations from user B additionally.

A reference user can be maintained in transaction SU01, but there could be some hurdles to avoid or restrict this possibility (for example, SAP notes 330067, 513694). These hurdles focus on maintenance of the reference user in SU01 and SU10. But even if it is not possible to assign a reference user via SU01 and SU10, it still works in Sc4SAP via the "Update table data" function as well as via maintaining the table using the "Maintain (edit) table data" function.

For getting the authorizations of the reference user being effective the user has to log into the system again. At login the system considers the given reference user for all upcoming authorization checks during the session.

Finally, I would like to mention that these possibilities (especially the 2nd and 3rd) should be used with caution. And: please do not forget to tidy up again. Only temporarily needed permissions should be revoked when they are no longer needed, and permanently needed permissions should be assigned in a regular way.

But in the end it is much more important that your go-live is back on track resp. your production system is running smoothly again, isn't it?

First contact

Shortcut IT — Tue, 24 Sep 2019 13:03:00 GMT

This article is intended as an introduction about the use of "Shortcut for SAP systems" and how to do the first steps.

Sc4SAP can be downloaded from our web page, it comes in a zip file. No installation is necessary, just copy the contents of the zip file in a suitable directory. I suggest to put it not in the same folders in which the applications normally are installed in because we need write access to some subdirectories, and that might be a problem using the typical folders like C:\Program Files (x86) or C:\Program Files.

It works fine even in a folder C:\Sc4SAP, which I used for this article.

Ok, so what we have got here? Maybe a look into the File "System requirements..." might be a good idea...

... Java Runtime Environment. Ok, got it on my computer.

... SAP Java Connector (JCO). Hmm, this is usually not installed on a standard computer.

... an accessible SAP system. Of course, that's why I have downloaded the program.

... an SAP GUI. Indispensable for my work, of course also available on my computer.

... ah, no transport requests, AddOn's or similar to be installed in the SAP system(s). That's nice!

There is a hint to use the check program "Sc4SAPStarter.exe", so let's try it.

That's what was expectable from reading the system requirements, the SAP Java Connector files are missing. SAP customers and partners can download it here. But maybe the files are already somewhere on my computer or an accessible network drive, thus, press the button "Search for it...".

I'm a lucky guy, the file was found. With the "Copy file" button it can be copied into the Sc4SAP subdirectory.

After closing the dialog the check procedure starts again. We are one step further now, but there is still something missing. Another missing JCO file:

Ok, once again I press the "Search for it..." button. The missing file was found in the same directory as the one before, and I also copied it. Now the check program does not complain any longer and after a short time the "Shortcut for SAP systems" application is started.

First greeting with a popup window about connections:

Connections... aren't there still enough in my SAPLogon?

There are, but the use of connections to SAP systems in "Shortcut for SAP systems" includes the specification of a user - different from the SAPLogon. Thus, let's do what is required. After pressing OK a click on "Settings and configuration" a maintenance function for the connections can be used. So I entered data to a local SAP system that I have installed on my computer, just to try it out and see around what's there.

After pressing the "Add" button the data will be moved to the (up to now empty) list of the maintained connections.

By pressing the "Test" button I verify whether the data is correct. As there is sensible data in it the function can be protected with a password. If your computer can be accessed by other people, I recommend to do this.

Tip: you can have multiple connections to a single SAP system / client, for example one with a system user (which often have wide authorizations) and your own (dialog) user. Some functions need dialog capabilities. Having connections with both a dialog user and a system user will offer much possibilities!

I saved the connection data and closed the window. Back in the main window I see that all the functions, which were previously disabled because there was no connection maintained, are available now. By moving the mouse cursor to a button I get some information about the function in the right part of the window.

Let's try some functions, starting with the first one "Get a list of users". This does not sound spectacular and offers no information that I can not get by using the functions directly in the SAP system, nevertheless I get an impression how the program works in general. In the window I can see the users in my local SAP system and some messages at the bottom.

Let's try another function, maybe viewing some data of the system. After chosing the function "View table data" and entering a table the application get in contact with the SAP connection and verifys whether the table exists. The button right from the table name offers some information.

Reading the data works for the given table:

Ok, nothing spectacular until now. Only positive aspect is that in case my own dialog user does not have authorizations for viewing some data (or I don't have a user in the client) I can use this application and a connection with a system user.

Let's switch to another function. Setting passwords I always find a boring task and there is a function related to this.

There are some gaps in the SAP functions for setting passwords:

There is no function allowing you to assign the same initial password to lots of users. This would be useful for example in case you have lots of test users. For each and every user the password has to be set individually. This is annoying.
There is no function allowing you to assign individual passwords to lots of users. It's the same mess as in point #1. Especially in case of system users after a system copy it is essential to assign the same passwords as before, otherwise the interfaces would not work. There is usually a list available with the user ID's and their passwords, but each and every system user has to be maintained individually.

The "Set password" function in Sc4SAP looks quite promising and seems to close these gaps.

Let's try it, but I prefer to do this not in the currently used client 000. So first I have to switch back to the maintenance function for the connection and create a connection to client 001. Done in few seconds: I click on the connection to client 000, the values are copied to the input fields below the list, I change few input values and press "Add".

I switch back to the "Set password" function to try it. At this point - unless you don't have a license for the application - you will be faced with this annoying popup:

So you have to have a license for this function (like for all other functions too, that are valuable). You will find the necessary information for ordering a license in the "Settings and configuration" area in the "Licenses" function. Of course it is possible to get a trial license for free.

But I am a lucky guy and have a license, so I don't get to see the popup window and can do what I wanted to do, i.e. provide many users with a new initial password in one go. Via copy and paste I fill the text field with the user ID's and entered an initial password. Pressing "Execute" and... wow, quite fast, after a second it is done!

Now let's try the other option: set individual passwords for lots of users in one go. As I mentioned before a good use case for this is the assignment of the correct password of the system users after a system copy. This could be really annyoing. Mostly you use complex, generated passwords (if you don't do this so far, you should do it), and this means you have to copy and paste the password for every system user.

The function in ScSAP offers 2 text fields, left one for the users, right one for the passwords. Of course the amount of users and passwords has to be identical. Filling the text fields is done fast using Copy and Paste (assuming that there is an Excel file or similar containing all the system users and their passwords). Pressing "Execute" results in the same fast action as for the previously tried password function.

Meanwhile, because I did not close the used functions, my desktop is quite full with windows.

There is no restriction to use another function while others are still running or open. You can start other functions, e.g. to use them for other connections or other purposes, it doesn't matter. All of them operate individually, and you can start and use them as often and in parallel as you want.

Also this article got quite long meanwhile. Since this article was only meant to show the first contact with the application, I will not continue in detail for all other functions like this. But I want to give some general information which might be useful for you.

Most of the functions offer the possibility to save your input in "variants". If so, you will find 2 buttons at the top right position of the dialog window:

Surely you know this concept from the SAP systems. It's not just that you don't have to enter the same values over and over again. Another advantage of the variants is the possibility to use the functions with the command line tool that comes with our product. The command line tool allows you to use the functions - outside of the GUI application - at the operating system level, and using them at the OS level in turn allows you to use the functions in automated workflows. You can find more information here and here.

When it is about transferring a file between your computer and the SAP system the SAPGUI might come up with a confirmation dialog. Until there is no confirmation the task will not continue. Sometimes the window does not appear visible in the foreground and is indicated by a blinking SAPGUI task in the Taskbar only. The confirmation dialog is not specific to Sc4SAP, it's part of the security concept of the SAPGUI.

For not to be faced again and again with this dialog for every file that should be transferred between the SAP system and your frontend you can add a Security Rule in the SAPLogon settings. You can click the "Help" button to view SAP's information about these settings. To shorten it: select "Options..." in the SAPLogon, > "Security" > "Security Settings" > select "Customized" in the "Status" listbox > click on button "Open Security Configuration" > button "Insert" > add a rule with Type = "Directory", Object = your preferred directory, e.g. "C:\Sc4SAP\data", Action = "Allow", Access Types = "Read, write" > click on buttons "Insert" and "OK". Once this rule has been added, you will not get any further confirmation dialogs when uploading or downloading files from this directory.

The "Messages / log" area will be fed with information of your started task. Sometimes this information is not important, but sometimes it is. There are few visible lines only. In case you want to see more just click the button at the right over the "Messages / log" area and the lines will be shown in a separate window, which can be sized as you want.

In case you closed a function, but you need the information of the messages: it is not lost. All messages will be stored in the "logs" directory for enabling you to follow up what you have done so far and also get all the information that came up in the functions.

Nearly at the end of this article I would like to refer to my personal favorite functions, I think they are very, maybe most valuable of all functions.

The "Copy user" function allows you to copy users on-the-fly between systems. As the user ID's can be given (or even specify "*" for all users) it is extremely flexible. And it offers also the possibility to store the user data in a file and use this file afterwards as source. Especially keeping users of a system that is to be overwritten via system copy can be done easily using this function: store the user data in front of the system copy in a file, afterwards use the file as source and copy the user back into the system. You don't have to bother any longer with client copies to temporarily created clients to park the users there! And using the command line tool it can even be integrated in automation tasks.
Also in a roll-in project or a client merge when it is to consolidate one or more clients into one (or the other way round in a carve out project) this function ist totally useful. I cannot imagine any other method that is so simple and flexible!

The "Process table data using R3trans" function is extremely useful for dealing with table contents in a system or to copy / move data to other systems. Among other things, you can use it

to copy data to another system or client (for example some customizing data). Have a look at this blog article.
to keep a backup of system specific tables (e.g. RFC connections, partner profiles etc.) and getting more security of the operation of your SAP system by this (there is a blog available about this topic)
to keep data of a system that will be overwritten via system refresh (again: RFC connections etc.). Especially with the command line tool this can save huge effort and duration time. Have a look at this blog article.
to enable the project to repeat a test migration without having the need to do another system copy again (think of saving the customer tables KNA1, KNB1, ... in front of the test migration and restore them afterwards if necessary). It does not have to be a big test migration, you can also use this function for very individual program tests. If your program modifies table xyz then make a backup of this table and restore the table after the test to its previous state - ready for the next test! Don't bother with additional programs, individually to be developed, to get the tables back to an "untouched" status.
to get information about 3rd party transport requests before they are imported in the system, using the "List data file content" action. As far as I know there is no other method to do so (and as far as I know no other product offers the "R3trans" functions to use them for other purposes than the transport stuff). Typically a sandbox system is to be set up for 3rd party transport requests, or you just take the risk to import a 3rd party transport into your system without knowledge about what's inside.
to get access to a new client on a system, by copying the user tables into the new client (see this blog about this topic).

The "Download / upload table data" is extremely useful for copying tables partly (selected records) to another system / client or saving / restoring data in the context of a system refresh. For example, in this blog article is described how to copy Idocs from the Production system to the Quality / test system.

Also the other functions for modifying data in the "View / modify data" section can safe a lot of time and money in your project work as well as in the IT support. Correction of data could become very cumbersome. With these functions it can be done really easily and fast and can be your rescue in time-critical situations. But of course they should be handled with care! Otherwise your data correction would end up in data corruption... :-(
The program comes with a command line tool. This makes it possible to execute nearly all functions outside the GUI on OS level. Consequently also a use of the function in automation scenarios is possible. This offers a wide variety of use cases, e.g. for system refreshes, data backup, user distribution and many more.
There are 2 methods available to use the command line tool: you can supply the function and a variant (= variant in Sc4SAP) as arguments or you call the command line tool with an XML file as argument. Whereas the first method requires a variant, the second one offers a lot of flexibility and possibilities. Have a look at this blog.

Copying users

Shortcut IT — Tue, 24 Sep 2019 10:16:00 GMT

The "Copy user" function in "Shortcut for SAP systems" offers a lot of possibilities. It is very handy, quite fast und very flexible. Here are some use cases that can be covered with this function.

Your test system will be refreshed via system copy from the productive system. With the system copy all users on your test system will be deleted resp. replaced with the users of the productive system. But... some test users will get lost, also the members of the project team, the system users will be the wrong ones, which would result in non-working interfaces, and last not least you will have also the much more restrictive authorizations on your test system. With that situation a continuation of the current test would not be possible and a lot of time and effort would be necessary to add the missing users and the extended authorizations.

Ok, of course it's not the first time this happens, and meanwhile you are dealing somehow with it. Possibly you have the PCA tool in use, which enables you to keep users and authorizations in front of a system copy and replace that data afterwards again.

Or you use a client copy to somewhere in front of the system copy (to another new client on another system) and do the same in the other direction afterwards. But it's all very elaborate, cumbersome and time-consuming, and you are under pressure because of the tight schedule of the project (as ever), unfortunately sacrificing another weekend :-(

Or, that's simply not what is needed. For example, the project team wants to test under circumstances that are as near to the productive environment as possible, so using the users and authorizations from the productive system is a good idea in general. But of course there are exceptions: special test users were created, and also the system users are different from those in the productive environment. So, there is still the need to adjust the users.
For every new system or client you have to create the administrative users. Depending on your organization and the role of the client (000 or a client with business data) this could be more than only a small number of users. And, btw, creating them one by one in SU01 is boring.

Also this problem is not new. You can think of solving it via a client copy from another client that already keeps the administrative users. For this you have to create an RFC connection on the source system and perform a client copy. Also a CUA (central user administration) system would offer functions in this area - accompanied by the usual inconveniences and problems of a CUA... Or maybe you have an Identity Management System offering functions for this. However, it's always a lot of effort.
You are faced with a roll-in project or merge of systems or clients, and users from multiple clients have to be migrated to a single target client.
Or, just the other way round, in a carve-out project users selectively are to be migrated to another system.

So let's have a look at the "Copy user" function. The most valuable features of it are in my opinion:

you can specify the users you want to process. This is completely different from the SAP client copy, where no selection is possible.
you can use a file as target as well as the source. This means that for saving the users you don't have to create a new client on another systems, including RFC connections. Just use a file as storage for the users, and use it again as a source when you want to copy the users into a client. It works even with a huge amount of users (we tested it successfully with an amount of ~150000 users).

It is so easy. For storing all users of a client just a few steps are necessary and the users are stored in a file. Specify source and target (in this case a file on your computer), press the "Execute" button, that's it. In this example we use a file on our computer for storing the users, therefore an SAP connection using a user with dialog capabilities is needed. In case you copy the users from an SAP client directly to another one or you use a file on the SAP system you can also use a connection with a system user.

In this example only a small amount of users are in the source client which of course results in a small data file. But also for clients with a huge amount of users the procedure works reliable. Data files will be splitted at a filesize of approx. 100MB, the filename will be extended by ".Partn" where n is incremented. Sc4SAP stores an ".info" file beside the data file on your computer. Keep data file(s) and ".info" file together in the same directory.

Now let us use the file to copy them into another client (here: 100). We select the file as the source and address the client 100 in the system as the target.

For creating the file we didn't restricted the amount of users and took all ("*"). Sc4SAP now offers all users for which is data in the selected file. Having a look at them there are some I don't want to copy because they already exist in client 100, therefore I leave the "Overwrite" flag empty and don't change the user ID's in the text field. But the possibility is there, and in case you want to reduce the amount of users to be copied you can do that!

Thus, let's start....

Connection established, starting to copy... but errors came up:

Ok, have a look into the messages for what the user copy process is claiming about. By using the most right button above the messages we see it all a little bit more comfortable than in the small message area. Obviously it is a general problem with the address data.

In every user master data record there is an address number belonging to the company address maintained in SUCOMP. This data is client-dependent and the company addresses in the target client are not the same as in the source client. Therefore the creation of the relation between the user master record and the company address fails. In case you have set up the target client as a copy from the source client this error should not occur. But here we have a client with other company addresses.

For my purpose the company address is not important, so I decide to use the option to do the copy without the relation to the company address:

Now it looks better, the copy process is working and it's quite fast. After some seconds it is finished.

Now it looks fine. Already existing users were not copied, but the others were. If I had chosen to overwrite users, the user used in the SAP connection would be excluded from the set of users to be copied! The copy process runs with the user used in the SAP connection, and overwriting that user would cause a temporarily loss of authorizations, therefore the copy process would fail. If you choose to overwrite existing users and the specification of the source contains the same user as used in the SAP connection, you will get an information about this.

But in this example we did not overwrite users and only added non-existing users to our target client.

So finally 59 users were copied successfully to the target client, and it took only few minutes. I did not have to struggle with a CUA or an Identity Management system, I did not need to maintain RFC connections or wait for others to maintain RFC connections. I can use the same data file to copy the users again, possibly into the same client after a system copy or another client on another system. And it can also be integrated into automated processes via our command-line tool (you can find some information in this blog article)!

But there is still one issue: the passwords. Maybe you recognized that there is an input field in the copy function for the initial password which I filled with a value. In case you leave it empty a password will be generated (you will get the individual passwords in the messages). In case you missed this somehow: there is nothing that can not be done afterwards with the "Set password" function which gaves us the possibility to assign the same initial password or assign randomly generated passwords.

"Shortcut for SAP systems" uses the function modules from SAP which are intended to deal with the users. To name the most relevant here: BAPI_USER_GET_DETAIL (for reading the user from the source client) and BAPI_USER_CREATE (to create the user in the target client). The function modules don't offer the possibility to copy also the passwords.

If it's ok that the users will have either the same password (as given in the "Copy user" function) or a randomly generated password, then the job is finished here. The users exist in the target client and it took only few minutes. We can use the same data file for other clients, either on the same or another system. This procedure looks suitable e.g. to create a base stock of basis and system admins and some system users on every new client and can be carried out easily and fast.

If your aim is that the copied users will have the same passwords than in the source client we have to do an additional task: copy the USR02 entries for the copied users (the USR02 table is an essential table for the users on the system and contains the individual passwords, stored in hash values). This can be done with the "Download / Upload table data" function. For avoiding that the password will be changed for users that will not come with the data file we reduce the download to the specific amount of users by giving a "where" clause.

After pressing the "Execute" button an SAP screen comes up with a confirmation message. For continuing your work in Sc4SAP you have to press the "Back" button in the SAP window, otherwise Sc4SAP does not recognize that the SAP system has finished its task and still waits for the completion.

For copying the passwords of the users to the target client we now have to use an SAP connection to the target client and use the "Upload" option. After pressing the "OK" button in the appearing confirmation dialog the data are uploaded to the target client.

After 2 seconds an SAP screen appears:

Ah, ok. There are already USR02 records for the 59 users, and overwriting them is exactly what I wanted to do. I forgot to set this flag in the dialog:

After setting this flag and a new execution I received what I expected:

The users in the target client now have the same passwords as in the source client.

Consider that, due to the fact that we copied the USR02 records, also attributes like the lock status have been copied. Therefore there might be the need that you have to unlock users.

I don't want to finish this article without giving the warning that the function for downloading/uploading and deleting table entries is very critical! Please, please be quite sure what you are doing with this function.

Get into a new client without struggling with SAP*

Shortcut IT — Tue, 24 Sep 2019 10:14:00 GMT

This article is related especially to the requirement for a new client on a test system. On productive systems it's about mostly one client only, and on a development system the requirement for new clients is seldom. But for test system the request comes up quite often.

Ok, so what is the problem...

Creating a new client in SCC4 is done quite easily and quickly. But then... how to get into the newly created client? The client is empty, there is no data and of course also no users. In this case, SAP plans to use the hard-coded SAP* user, which can also be used to log on, even though it does not actually exist.

But in general the possibility of using the SAP* user should be avoided, it's a kind of an open door to the system. Therefore for avoiding the access to the SAP* user there is a profile parameter login/no_automatic_user_sapstar which is by default set to 1 which means: no login using the (non existing) user SAP* is possible. Of course you can change this profile parameter to 0 but this profile parameter is not dynamically switchable and a restart of the system is necessary for taking effect of the changed value. Afterwards you could use the SAP* user, but of course the profile parameter should be switched back to its original value and another restart is necessary.

Often you will find this situation regarding the SAP* user (which is a good method, I support this):

profile parameter login/no_automatic_user_sapstar set to 1 which disables the login with a (non-existing) user SAP*
user SAP* created in the clients, but locked by admin and without any authorizations.

And this points us into the direction that we need to change the profile parameter in the profile editor and restart the system. Hmm, cumbersome... in case the system is currently in use you have to deal with the stakeholders about a downtime (or more precise: 2 downtimes) and of course there will be the usual side-effects raised from a system restart like failed jobs, failed update tasks etc. Maybe the project leaders raise the idea to do this at the weekend. Congrats!

Another possible method would be to do a client export of an already existing client of your system (transaction SCC8, profile SAP_USER). This would create 3 transport requests which could be used for import into the new client. However, the import has to be done differently from the usual procedure because umode 18 is necessary which can't be set via STMS. Thus, you have to use tp via OS command.

Ok, we see it's of course not impossible, but there should be an easier and shorter way to deal with the access to the new client. Let's see whether we can shortcut this...

Yes, of course we can, otherwise I wouldn't have written this blog ;-)

Here is a step-by-step guide.

Let us have a look again at the "Process table data using R3trans" function and the tables related to users. Same as in the blog about saving system specific tables we can use the PCA tool for getting the information about the tables. We use the ABAP program that comes with "Shortcut for SAP systems" and take all tables of category 'USER':

Though the number of 135 tables seems to be a big amount: as we use client 000 in this example the amount of data is not so big, and exporting the data needed less than 10 seconds. Notice that in this example we defined that the data file is to be stored on the frontend, our computer. This requires an SAP connection that uses a user with dialog capabilities. Of course you can also use the server as the location of the data (and log) file, in that case you can use a system user. In this example I use the frontend and a connection with a dialog user.

A look into the log file offers some details about the exported data:

So, now we have a file with R3trans data containing users. And, as there were also the AGR_... tables given we also have the roles in it.

Now do the import of this data in our new client (here: 100). There are only 2 input fields to be changed: switch from "Export" to "Import" and use the target client, now. We do not need a connection to the target client (which is not possible of course, because there is no user up to now - that's what we want to change here). We can still use the same SAP connection we used for the export. Only the "Client" field has to be changed to the target client.

Pressing on "Execute" Sc4SAP comes up with a confirmation dialog, containing also an information that might be of interest:

Okay... that is a point we did not consider when we did the export. There is the possibility that importing also the client-independent data could have some effects to the other clients...

Sc4SAP gives the information of client-dependency for every table entered in the input field and did this also when we entered the tables for the export. Here we are going to do the import into the same system, so it would not make a difference. However, it is not a big issue to consider the client-dependent tables only and it's still not too late to do so. So let's stop here for the moment, cancel the confirmation dialog and reduce the tables in the input field to those which are client-dependent. We can use a text editor for this (here: Notepad++, free available and useful for lots of things). Copy the messages of Sc4SAP regarding the client-dependency into a new window and select the lines with "client dependent".

After pressing the button "Find All in Current Document" a 2nd window comes up with all hits:

A right mouse-click, followed by "Select All" and the same for "Copy" copies all 124 tables into the clipboard. In a new editor window we can paste the lines and reduce the data leaving the table names only (simply use the "Replace" function to replace all texts "Table " and " is client dependent." with nothing). Then switch to Sc4SAP and replace the content in the "Tables" input field with the reduced amount of (now 124) tables (maybe it sounds a little bit cumbersome, but it is a task of a minute only). And let us do the export again. It would be also possible to reduce the amount of data to the client-dependent tables for the import. The other tables would not be processed. But I would like to have this tidy for later requests of new clients for the system, and doing the export is a task of few seconds only. Spending these few seconds for repeating the export will preserve me in future of dealing with the separation of client-dependent and client-independent tables when I have to use the same data file later again (for other clients). Thus, switch the R3trans operation back to "Export" and client "000" again and repeat the export. See the screenshot above in this article.

After having created the data file with client-dependent data only we use "Import" and the target client to create user and authorization data in the target client. We will be asked for confirmation again, but now without the warning about client-independent data:

Yes, we are sure, and pressing "OK" starts the action.

It takes noticeably longer than the export (the import of data into the database causes more effort than reading), but after approx. 2 minutes we see the feedback from the SAP system. Hmm, a warning...

Thus, let's have a look into the log file for the warning(s). By pressing again the button with the glasses the log file will be shown in a separate window. The warnings in the log file of R3trans have a "W" in the 2nd column (the first column is a log level that is used in the SAP system for expanding and collapsing a log). There are 2 warnings in it:

The first warning tells us that we specified the tables to be imported. R3trans allows also to leave the naming of the tables blank, in that case R3trans take the whole content of the data file. We specified the client dependent tables only, so the statement about the 'selective import' is true, but we can ignore the warning. The statement of the other warning also is true. We used the same system to transfer users and authorizations from client 000 to client 100.

Ok, finally we are at the end. Now check if we can log into our new client. We copied all the user tables from client 000 into client 100, therefore we expect to have the same users and the same passwords:

Tataaa.... we logged in.

Looking a little bit around in our new client offers no big surprises. We find the same users and the same roles in our freshly created new client. We did not have to deal with profile parameters and restarts of the system. And although this article got quite long and extensive, the time necessary for doing it was quite short and the tasks can be done in a few minutes. And: the data file can be used again for getting into another new client on the system or a new client on another system. This could be done now in a few minutes, without the need for profile parameter changes and restarting the system!

In a 2nd try I reduced the tables. As I chose client 000 as source, containing users (also mine) with profile SAP_ALL, it is not necessary to copy all the roles. It should be enough to copy the USR* tables (containing the users) and the UST* tables (containing the authorizations, e.g. the SAP_ALL profile), so I decreased the amount of tables to these:

USGRP; USGRP_USER; USGRPT; USR_CUST; USR01; USR02; USR04; USR05; USR06; USR06SYS; USR08; USR10; USR11; USR12; USR13; USR14; USR22; USRACL; USRACLEXT; USRATTR; USREFUS; USREXTID; USREXTID_IDM; USREXTIDH; USRSTAMP; UST04; UST10C; UST10S; UST12

The result was the same: I could log into the new client with my user credentials from the source client.

So, until now we shortcut the login into the new client, but - except some tables for user and roles - all the important tables are still empty (addresses, number intervals, country settings etc.etc.etc.... in short: the whole customizing). Without this data not much more than a login into the client is possible - which was the target we wanted to achieve, and we did it.

Therefore, after now having the access to the new client you can (and have to) continue like you would do as usual, e.g. with performing a client copy in SCCL using profile SAP_CUST (Customizing). Means: enable now the SAP* user (mostly it is created in the systems with respect to the security gap related to this user, otherwise you can create the user now), assign SAP_ALL and use the SAP* user to perform a client copy with the new created client as the target.

About security and passwords

Shortcut IT — Tue, 24 Sep 2019 08:11:00 GMT

There are many sources of information available about SAP's password security, therefore why add another one...?

"Shortcut for SAP systems" claims - among other possibilities - to work as a "locksmith" in case you need it, therefore I thought it is worth to spend some time in an article about this. There are some functions related to this topic in the application.

The "Unlock users" function can be used for the task of unlocking lots of users. In case you are already a user administrator it is only an alternative to the SU10 transaction, otherwise the function enables you to unlock users using another user (e.g. a system user).
The "Unlock user via DB update" is a little bit more special, it offers the possibility to unlock users in another client of the system. In case every user is locked out of the system by mistake (or all user administrators) this could be the rescue.
The "Set password" function closes some gaps in the SAP standard functions and makes it easier to assign passwords to lots of users.

With these functions we can unlock users in every client and assign passwords for the same client as in the used SAP connection. This leaves us with the gap of assigning passwords in every client. In other words: how to assign a password in a client to which we don't have access?

You might think that this is not possible. But... it is! As there is no access to the client we can not use any function modules or similar to assign a free chosen password. But we can try it the other way round: update the USR02 table with a hash value for a known password! Of course we need access to another client of the system.

This method needs a little explanation, but from technical and security point of view it is quite interesting!

There is a function are that can be used for this. Beside all other tables the "Update table data" function also enables us to update table USR02.

Some background info about the method how passwords are stored and verified at login:

The hash functions used by SAP for the passwords have a long history. Started with simple hash methods for quite short passwords (up to 8 ASCII characters, automatically converted in upper case letters) the hash methods in the recent releases are meanwhile quite secure, dealing with special characters and a maximum password length of 40 characters. There's some "salt" given, means that something randomly generated is appended to the password and that concatenated value is used to compute the hash value.
The method used by default in the system depends on the SAP kernel version and some profile parameters (login/password_charset and login/password_downwards_compatibility) and can be seen in the USR02 record in field CODVN. They were added and used by SAP evolutionarily. For compatibility reasons the SAP system supports the newer hash methods as well as the older and outdated ones.
The hash algorithm can vary from user to user. When a user logs in into the system, the hash algorithm stored in the field CODVN of the USR02 record belonging to the user ID determines which hash algorithm and which field containing the hash code is to be taken for verifying the password.

When using the "Update table data" function for setting passwords, we need some knowledge about the internals. That's why I would like to make a short excursion into the dry topic of password encryption.

Depending on the hash algorithm specified in field CODVN the hash value is stored in different fields of the USR02 table. 'A' to 'E' uses the BCODE field which is long enough to keep a hash value for a password with up to 8 characters. With 'F' a password length up to 40 characters is supported, the hash value is stored in the field PASSCODE. The complexity of the hash algorithm made a further step with CODVN 'H' which stores the hash value in the field PWDSALTEDHASH.

For our current purpose - assign a password to the emergency user in a client where we don't have access to - the hash algorithm behind CODVN 'H' is quite interesting. What? This is a quite modern hash algorithm, much newer than for example 'B'. So, with respect to the availability of this old hash method 'B' and its lower level of security, why should the modern 'H' algorithm with a higher security level looks more attractive to do some tricks?

There is one aspect that makes the 'H' algorithm useful for our purpose: there is no dependency to the user ID in the hash algorithm - different to the old and outdated 'B'. If we would copy the password information for the 'B' algorithm from user A to user B, the password would not work for user B because of the different user ID considered for computing the hash value. Therefore, copying a password for CODVN 'B' or other hash algorithms considering the user ID from a user to another user ID would not work. It has to be the same user ID.

The 'H' algorithm uses a randomly generated "salt" for the hash value. You can see this when you assign a password to a user and have a look afterwards on the field PWDSALTEDHASH where the hash value is stored (I assume that in your system the 'H' algorithm is used as it seems to be the standard algorithm in SAP systems since releases with SAP_BASIS 731). Do it a 2nd time using the same password and the same user and you will see that the value has changed. A 3rd time... again another hash value. That is the effect of the "salt": the field contains a hashed value of the concatenation of the "salt" value and the password, and as the "salt" is randomly generated the value is another one every time. But, and this is the important point: the 'H' algorithm does not use the user ID additionally for encryption.

Therefore, for our purpose (it's still about assigning a password to an emergency user in a client where we don't have access to) this hash algorithm, even that it has a really high level of security, offers an advantage in comparison to the old 'B' algorithm. Because the field PWDSALTEDHASH contains salt + password and the algorithm does not use the user ID, we can transfer the hash value to another user without losing its validity.

Seems to be too easy? Let's verify it...

I use my local SAP test system for it. In client 001 I created a user 'EMERGENCY', assigned a randomly generated password and forgot it. In another client I created a user 'TESTUSER' with initial password 'Shortcut!'.

Now let's copy the value of the field PWDSALTEDHASH of this user to the EMERGENCY user in client 001.

First get the value by using SE16 for table USR02:

I copy the value into the clipboard and start the "Update table data" in Sc4SAP to insert the values for updating the USR02 record for the EMERGENCY user in client 001, using an SAP connection for client 000.

The update of CODVN with 'H' is not necessary when in the USR02 record the value is already set. It ensures that the right hash algorithm is used for the verification of the password.

After pressing "OK" of the confirmation dialog the update needs only a second.

Now comes the thrilling part. We try to log in with the EMERGENCY user in client 001 with password 'Shortcut!', and... it worked!

It works on the same system as well as on other systems!

There is a relation to profile parameter login/password_hash_algorithm which specifies some parameters for the hash algorithm. Default value for this profile parameter is 'encoding=RFC2307, algorithm=iSSHA-1, iterations=1024, saltsize=96'. The SAP documentation of this profile parameter says: "This profile parameter is evaluated when calculating new password hash values (but not, however, when checking password hash values at logon), to determine the hash procedure and the coding format."

Therefore it should work also on systems with another value for this profile parameter.

If you want to try it, this is the used value for PWDSALTEDHASH with CODVN = 'H' that works for password 'Shortcut!'.

'{x-issha, 1024}fMSNQye6C9qWpeEdfyPwCFUkTfcBG3Sae4dthuxjfPA='

Following the given example feel free to try it with other passwords and their hash values. In addition to the update of PWDSALTEDHASH you can of course take the opportunity to update some other fields related to the login, for example:

UFLAG = 0 /* lock flag (0: not locked)

LOCNT = 0 /* counter of invalid login attempts

PWDSTATE = 0 /* password can be changed

PWDINITIAL = 2 /* password is not initial

LTIME = sy-uzeit /* no need to change password at 1st login

GLTGV = '00000000' /* } reset validity range

GLTGB = '00000000' /* }

PWDCHGDATE = '29991231' /* } bypass 'deactivated' password (see profile parameter

PWDLGNDATE = '29991231' /* } login/password_max_idle_productive)