In this post we will exclusively see about how to configure a new data page and its scope.
- Every application needs some data to support its processes.
- Basically data resides in memory, database, or other external applications.
- Pega uses clipboard data structure to store the data in memory.
Scenario: Imagine some insurance call center’s CRM application is using Pega. All the customers information stored in different applications and policy details are stored in Pega DB.
So when a customer calls in call center for an enquiry, we may need to use connectors to connect with other applications to get customer data and use database connection to get the policy details.
When the call center agent processes the enquiry, we need to access the customer and policy data again and again from other applications.
Yes, this is very time consuming and brings performance degradation.
Solution: So instead of accessing data from DB or other applications, we can pre-load the data in memory and improve the performance.
Here we can use data pages D_Customer, D_PolcyDetails to pre-load the data in clipboard and use it whenever necessary.
When we refer the data page in any rule, the data page gets loaded in the clipboard. On subsequent access, the data pages never gets loaded, since it can use the same page in clipboard.
Refresh strategy can be implemented. We can see more in detail at the end.
Data pages can be distinguished based on their scope. There are three types
- Thread level
- Requestor level
- Node level
As we saw in the ‘Requestor types’ () post, we know that all the clipboard pages are saved in memory.
Node – Pega system.
Every requestor logs into a particular node. Each requestor can contain multiple threads opened.
Just assume that we are having this structure as a folder type like (c:\Program file\citrix)
Node \ Requestor \ Thread.
Node Level – If you specify a page in node level, then it can be accessed by all the requestors in the particular node.
Requestor Level – If you specify a page in Requestor level, then it can be accessed by all the threads opened by the requestor.
Thread Level – If you specify a page in Thread level, then it can be accessed only by that particular thread.
Hope you got the basics. Now, let us see in detail.
What is a data page?
- Data page belongs to data model category.
- Data page can be of single page or a page list with multiple pages.
- You can access the data page in clipboard.
- Data pages starts with D_ or Declare(Legacy support).
- You can pre-load the data page as well as clear the data page.
- Data pages can be editable/ Read only.
- Data pages can accept parameters.
How to configure a Data page?
You can create a data page for any work class or data class.
Let’s create a data page for policy list – D_PolicyList
There are 3 main tabs.
1. Definition tab
Data Page definition
- Page – This data page can be treated as single page structure. Ex: D_Policy refers to single page with single policy details.
- PageList – This data page can be treated as a pagelist containing multiple pages (Code-Pega-List) results. Ex: D_PolicyList refers to many policies embedded into a pagelist.
b) Object type – Specify the class of the data page. It can be either work class or data class.
c) Edit mode
1. Read only – As the name implies this datapage is pre-loaded and cannot be updated. All node level data pages are read only.
You can see that if you select editable, then scope will be limited to thread and requestor.
You can find the read only data pages under ‘Data page’ category, whereas editable data pages are in ‘User pages’ category.
2. Editable – Data pages can be editable. You can find it in ‘User pages’.
- Thread – Scope of the data page applies only with in a particular thread. In the above pic,
Thread – New expand the data pages -> Thread level. You can see D_pzConnectorDataSourceTypes.
Now let us switch to different thread ‘standard’ and check the data pages. You will find different data pages.
What is keyed page access?
- This is applicable only for Pagelist structure and mode – read only.
Remember, we have a Datapage list D_PolicyList which contains details for all the policies for the customers.
We can use the same data page to get the details for a particular policy.
Example: Imagine you need to a check a particular chapter in a book. If you know the page number, then you can directly switch to that chapter right. This is called ‘keyed page access’.
Like the same, you have D_PolicyList(book) contain different policies (chapters). If you need to access a particular policy, then you need to know the policy ID (page number) to access.
Keyed page access
- Access pages with user defined keys – Checking this will enable you to access particular pages using keys.
- Allow multiple pages per key – Checking this will allow you to return more than one page using keys. Just think single policy ID can return multiple policies. (I am saying this, just for example).
- Pagelist Keys – You can specify multiple keys to access.
- You can make a single data page to be used as both pages or pagelist structure.
D_PolicyList can return all polices, also keyed page access can return single policy.
- Avoid multiple rules management.
Data pages should contain data. We can specify the data sources here.
Data sources vary based on the page structure.
- Connector – You can specify the connector type with connector activity. You can also specify the request and response data transform, end point URL.
These data pages act as a primary page for the connector rules and can contain the response within the data page.
2. Data Transform – You can specify a data transform rule to map the data to the data page rule.
3. Lookup – You can specify a class to get a particular class instance. You can also specify a response data transform. Appears only when data page structure – Page
You can get the details of a Service Request work item in a data page, by passing the work item ID using look up option.
4. Activity – You can specify an activity rule to load the data page with data.
5. Report Definition – You can specify a report definition rule to get the results and load the data page. Appears only when page structure – Pagelist.
Simulate data source
During development, when data source is not ready, you can simulate the data source for testing.
Check the simulate data source on right corner.
You are having three options to simulate the source.
Data transform, Lookup, Activity can be used to simulate the source.
Post Load Processing
You can specify a post load processing activity to process the data page after loading.
This is mainly used in error processing. Remember if the data source returns invalid data, then we can have logic here to process the errors.
2. Load Management Tab
Page Management – Clear Data page
You can click on clear data page to remove the data page from the clipboard.
Access group – This is applicable only for node scope data page. Specify the access group here.
We know rule resolution works based on the requestor access group.
But for node level, you may have different requestors with different access groups per node. So we need to specify an access group here. Pega uses this access group to identify the data source activity using rule resolution and load the data.
Refresh Strategy for node level data page
Reload if older than – You can specify the days, hours, mins, secs constant value to expire the data page.
Imagine you are using a data page to connect to different application to get security token. Business requirement will be like the security token is valid for only 8 hours. You need to request the application again to get a new security token after 8 hours.
Here we can create a data page D_Authorization, which can use connector as data source to get the security token.
You need to specify hours as 8 and leave other fields empty in refresh strategy.
So after 8 hours, this data page gets expired. So after 8 hours, if you refer the data page again, then data page gets refreshed with new data.
Refresh Strategy for thread/ requestor level data page
Reload once per Interaction – Checking this will allow data page to get loaded again and again when referred. This actually destroys the advantage of data page.
Do not reload when – Specify a when rule.
When true – Use existing data page. No reload.
When false – Data page gets refreshed on subsequent access.
Page Limits – Clear pages after non-use
We know data page resides in clipboard even after expiration. Yes, it consumes space.
For node level
- You can check ‘clear pages’ after non-use to remove the data page from clipboard.
- DeclarePages/DefaultIdleTimeOut – DSS set to one day. After which the idle node level data pages gets removed.
- So after 1 day all the expired node level data pages get removed.
For requestor level
- Requestors can open multiple threads (I can open multiple workitems), so when a thread is requesting a requestor data page to get removed, then requestor level data page also gets removed.
3. Parameters tab
You can specify a parameter for a data page. For D_PolicyList we can specify a PolicyID parameter.
This is more or less like a keyed page access. We can load the declare page with particular policy ID.
After specifying parameter, you can refer the data page like D_PolicyList[“12345″] / D_PolicyList[PolicyID:”12345”]
Data page example
Lets us see an example using the Countries data type.
Step 1: Create a data page D_Countries with the below configuration.
Step 2: Specify the source as Lookup to TVS-Data-Countries. Pass the parameter as
Step 3: So, where do we specify this parameter? Yes, we specify it in parameters tab.
Step 4: Let us run the data page and check if it is available in Clipboard -> Data pages -> Thread, since it is read only.
You can test by varying the scope and edit mode and check the clipboard to see the availability.
Where can we refer a data page?
Data pages can be referred in activity, data transform, sections, properties repeating grid.
How do we load a data page?
- Whenever you refer the datapage for the first time in any of the above rules.
- In activity, you have a method ‘Load-Datapage’ to load the data page asynchronously.
What is asynchronous data page loading?
- You can specify a data page to load asynchronously in an activity using Load-Datapage method.
Imagine the call center CRM application. When the agent engages with the customer through the phone, he needs to get the customer details before him. We may need to integrate it with other applications to get those data which can affect performance during loading.
Here, we may prefer loading the data asynchronously using data page.
- When the Load-Datapage methods gets executed, the datapage gets loaded in background thread and the activity continues processing without waiting for data page loading.
- You specify a pool ID for reference in load-datapage method.
- After all your processing step in the activity, you can use Connect-Wait method at last referring the wait seconds and same pool ID we used in load-datapage method.
How do we remove a data page?
- In production environment if you need to remove a data page manually, then use SMA. You can select a data page and clear manually.
- In designer studio, you can open the data page rule and clear data page from load management tab.
- While using Page-Remove in activity steps, never use remove node level data page. It is not recommended.
- Node pages get removed when node shutdown and thread/requestor data page gets removed, i.e. when user session logs out.
- Use load management effectively to expire the data page at the appropriate time.
We are at the end of Data pages lesson. 🙂