Data Pages in Pega

Introduction

In this post we will exclusively see about how to configure a new data page and its scope.

  • Every application needs some data to support its processes.
  • Basically data resides in memory, database, or other external applications.
  • Pega uses clipboard data structure to store the data in memory.

Scenario: Imagine some insurance call center’s CRM application is using Pega. All the customers information stored in different applications and policy details are stored in Pega DB.

So when a customer calls in call center for an enquiry, we may need to use connectors to connect with other applications to get customer data and use database connection to get the policy details.

When the call center agent processes the enquiry, we need to access the customer and policy data again and again from other applications.

Yes, this is very time consuming and brings performance degradation.

Solution: So instead of accessing data from DB or other applications, we can pre-load the data in memory and improve the performance.

Here we can use data pages D_Customer, D_PolcyDetails to pre-load the data in clipboard and use it whenever necessary.

When we refer the data page in any rule, the data page gets loaded in the clipboard. On subsequent access, the data pages never gets loaded, since it can use the same page in clipboard.

Refresh strategy can be implemented. We can see more in detail at the end.

Data pages can be distinguished based on their scope. There are three types

  1. Thread level
  2. Requestor level
  3. Node level

As we saw in the ‘Requestor types’ (http://myknowpega.com/2017/05/13/requestor-types/) post, we know that all the clipboard pages are saved in memory.

Node – Pega system.

Every requestor logs into a particular node. Each requestor can contain multiple threads opened.

Just assume that we are having this structure as a folder type like (c:\Program file\citrix)

Node \ Requestor \ Thread.

Node Level – If you specify a page in node level, then it can be accessed by all the requestors in the particular node.

Requestor Level – If you specify a page in Requestor level, then it can be accessed by all the threads opened by the requestor.

Thread Level – If you specify a page in Thread level, then it can be accessed only by that particular thread.

Hope you got the basics. Now, let us see in detail.

What is a data page?

  • Data page belongs to data model category.
  • Data page can be of single page or a page list with multiple pages.
  • You can access the data page in clipboard.
  • Data pages starts with D_ or Declare(Legacy support).
  • You can pre-load the data page as well as clear the data page.
  • Data pages can be editable/ Read only.
  • Data pages can accept parameters.

How to configure a Data page?

You can create a data page for any work class or data class.

Let’s create a data page for policy list – D_PolicyList

There are 3 main tabs.

1. Definition tab

Data Page definition

a) Structure

  1. Page – This data page can be treated as single page structure. Ex: D_Policy refers to single page with single policy details.
  2. PageList – This data page can be treated as a pagelist containing multiple pages (Code-Pega-List) results. Ex: D_PolicyList refers to many policies embedded into a pagelist.

b) Object type – Specify the class of the data page. It can be either work class or data class.

c) Edit mode

1. Read only – As the name implies this datapage is pre-loaded and cannot be updated. All node level data pages are read only.

You can see that if you select editable, then scope will be limited to thread and requestor.

You can find the read only data pages under ‘Data page’ category, whereas editable data pages are in ‘User pages’ category.

2. Editable – Data pages can be editable. You can find it in ‘User pages’.

Scope

  1. Thread – Scope of the data page applies only with in a particular thread. In the above pic,

Thread – New expand the data pages -> Thread level. You can see D_pzConnectorDataSourceTypes.

Now let us switch to different thread ‘standard’ and check the data pages. You will find different data pages.

What is keyed page access?

  • This is applicable only for Pagelist structure and mode – read only.

Remember, we have a Datapage list D_PolicyList which contains details for all the policies for the customers.

We can use the same data page to get the details for a particular policy.

Example: Imagine you need to a check a particular chapter in a book. If you know the page number, then you can directly switch to that chapter right. This is called ‘keyed page access’.

Like the same, you have D_PolicyList(book) contain different policies (chapters). If you need to access a particular policy, then you need to know the policy ID (page number) to access.

Keyed page access

  • Access pages with user defined keys – Checking this will enable you to access particular pages using keys.
  • Allow multiple pages per key – Checking this will allow you to return more than one page using keys. Just think single policy ID can return multiple policies. (I am saying this, just for example).
  • Pagelist Keys – You can specify multiple keys to access.

Advantage:

  1. You can make a single data page to be used as both pages or pagelist structure.

D_PolicyList can return all polices, also keyed page access can return single policy.

  1. Avoid multiple rules management.

Data Sources

Data pages should contain data. We can specify the data sources here.

Data sources vary based on the page structure.

  1. Connector – You can specify the connector type with connector activity. You can also specify the request and response data transform, end point URL.

These data pages act as a primary page for the connector rules and can contain the response within the data page.

2. Data Transform – You can specify a data transform rule to map the data to the data page rule.

3. Lookup – You can specify a class to get a particular class instance. You can also specify a response data transform. Appears only when data page structure – Page

You can get the details of a Service Request work item in a data page, by passing the work item ID using look up option.

4. Activity – You can specify an activity rule to load the data page with data.

5. Report Definition – You can specify a report definition rule to get the results and load the data page. Appears only when page structure – Pagelist.

Simulate data source

During development, when data source is not ready, you can simulate the data source for testing.

Check the simulate data source on right corner.

You are having three options to simulate the source.

Data transform, Lookup, Activity can be used to simulate the source.

Post Load Processing

You can specify a post load processing activity to process the data page after loading.

This is mainly used in error processing. Remember if the data source returns invalid data, then we can have logic here to process the errors.

2. Load Management Tab

Page Management – Clear Data page

You can click on clear data page to remove the data page from the clipboard.

Load authorization

Access group – This is applicable only for node scope data page. Specify the access group here.

We know rule resolution works based on the requestor access group.

But for node level, you may have different requestors with different access groups per node. So we need to specify an access group here. Pega uses this access group to identify the data source activity using rule resolution and load the data.

Refresh Strategy for node level data page

Reload if older than – You can specify the days, hours, mins, secs constant value to expire the data page.

Imagine you are using a data page to connect to different application to get security token. Business requirement will be like the security token is valid for only 8 hours. You need to request the application again to get a new security token after 8 hours.

Here we can create a data page D_Authorization, which can use connector as data source to get the security token.

You need to specify hours as 8 and leave other fields empty in refresh strategy.

So after 8 hours, this data page gets expired. So after 8 hours, if you refer the data page again, then data page gets refreshed with new data.

Refresh Strategy for thread/ requestor level data page

Reload once per Interaction – Checking this will allow data page to get loaded again and again when referred. This actually destroys the advantage of data page.

Do not reload when – Specify a when rule.

When true – Use existing data page. No reload.

When false – Data page gets refreshed on subsequent access.

Page Limits – Clear pages after non-use

We know data page resides in clipboard even after expiration. Yes, it consumes space.

For node level

  • You can check ‘clear pages’ after non-use to remove the data page from clipboard.
  • DeclarePages/DefaultIdleTimeOut – DSS set to one day. After which the idle node level data pages gets removed.
  • So after 1 day all the expired node level data pages get removed.

For requestor level

  • Requestors can open multiple threads (I can open multiple workitems), so when a thread is requesting a requestor data page to get removed, then requestor level data page also gets removed.

3. Parameters tab

You can specify a parameter for a data page. For D_PolicyList we can specify a PolicyID parameter.

This is more or less like a keyed page access. We can load the declare page with particular policy ID.

After specifying parameter, you can refer the data page like D_PolicyList[“12345″] / D_PolicyList[PolicyID:”12345”]

Data page example

Lets us see an example using the Countries data type.

Step 1: Create a data page D_Countries with the below configuration.

Step 2: Specify the source as Lookup to TVS-Data-Countries. Pass the parameter as Param.SNo

Step 3: So, where do we specify this parameter? Yes, we specify it in parameters tab.

Step 4: Let us run the data page and check if it is available in Clipboard -> Data pages -> Thread, since it is read only.

You can test by varying the scope and edit mode and check the clipboard to see the availability.

Where can we refer a data page?

Data pages can be referred in activity, data transform, sections, properties repeating grid.

How do we load a data page?

  1. Whenever you refer the datapage for the first time in any of the above rules.
  2. In activity, you have a method ‘Load-Datapage’ to load the data page asynchronously.

What is asynchronous data page loading?

  • You can specify a data page to load asynchronously in an activity using Load-Datapage method.

Imagine the call center CRM application. When the agent engages with the customer through the phone, he needs to get the customer details before him. We may need to integrate it with other applications to get those data which can affect performance during loading.

Here, we may prefer loading the data asynchronously using data page.

  • When the Load-Datapage methods gets executed, the datapage gets loaded in background thread and the activity continues processing without waiting for data page loading.
  • You specify a pool ID for reference in load-datapage method.
  • After all your processing step in the activity, you can use Connect-Wait method at last referring the wait seconds and same pool ID we used in load-datapage method.

How do we remove a data page?

  • In production environment if you need to remove a data page manually, then use SMA. You can select a data page and clear manually.

  • In designer studio, you can open the data page rule and clear data page from load management tab.
  • While using Page-Remove in activity steps, never use remove node level data page. It is not recommended.
  • Node pages get removed when node shutdown and thread/requestor data page gets removed, i.e. when user session logs out.
  • Use load management effectively to expire the data page at the appropriate time.

We are at the end of Data pages lesson. 🙂

61 thoughts on “Data Pages in Pega

  1. hi prem can you explain about cascading relationship using data page.ex when i select country name (first drop down) i want to display state names(Second drop down) with screenshots.
    Thanks

    1. Hi Krishna, thanks for pointing it out. I didn’t explain the cascading example completely in the example. I will update the same post tomorrow.
      😊

  2. Good read- What is the context of reload when rule. Here is my requirement- I have policies list as data page. Whenever a policy is created updated or deletedbi want to refresh this list. How would I auto reload my data page.

    1. Yes..
      Do not refresh when – you need to specify a when rule.
      For the above requirement, when ever a property gets created or updated set some property to true.
      You can use when rule to check if it false. So now the when rule returns false.

      It means do not refresh when – false.

      See usually when you refer any data page, Pega have some inbuilt code to check these parameters. If do not refresh is false then it clears the data page. So the data page loads again with fresh data.

      Hope you are clear now😊

  3. Nice Explanation Prem.
    One question on Asynchronous data page.
    When we loading data page Asynchronously then again why we are going to use connect wait which inturns makes us wait and finally this is loading as Synchronous.

    Assume a harness have two sections and for both context page is using data Page.

    Now the first section datapage takes more time than second one .
    So here first one blocking second one.

    Is non blocking UI concept not existing anymore in Pega 7 ?
    https://pdn.pega.com/how-configure-non-blocking-ui-using-asynchronous-declare-pages

  4. Can you please tell me the difference between asynchronous data page loading and normal data page loading in some more detail?

    1. Hi Rahul, imagine you built a screen with 3 repeating grids. All 3 grids are sourced from a report definition or some connectors.

      Synchronous processing :
      You need to load the data pages one by one before the screen renders. You need to wait for each data page till it gets loaded. Time consuming.

      Asynchronous processing :
      You no need to wait till the data page gets loaded. You can refer all 3 data pages and then wait at the end to complete loading. Here data pages are getting loaded parallel. time efficient.

      You can achieve this in activity using 2 methods:
      1. Load declarative page – specify the data page name & pool ID.
      2. At the end. use Connect-Wait & refer the pool ID. So we wait for all 3 data pages which gets loaded parallel.

      Hope I cleared your doubt 🙂

    1. Hi Pankaj,

      Data page is just a page in clipboard. You can refer the data page, just like any user page. You can refer it in decision table columns like D_page. If the page is not in clipboard memory, then it will load automatically. If it is there already, then the existing data page is used.

  5. Good read!
    What is the main purpose of Editable data page, Which scenerio we will go for it? How we will acheive it.

    Tell me the exact scenario for editable, Read-Only is usual case.
    Please let me know. Thanks in advance.

    1. Yes. You can see that data pages are heavily used in Pega 7.
      Example 1: You can populate a grid using declare page.
      If you want to update the grid results, then you need to have that declare page as “Editable”.

      Example 2:
      Say for example, you have created 2 cases. You need to pass the details from one case to another.
      In this scenario, I will create data page to copy the pyWorkpage – Scope will be requestor. So this data page can be available to all cases (thread) within the requestor.
      In some situation, you can update the data page from another case.

      You need to use it appropriately 🙂

  6. Hi,
    Excellent Boss.
    Today one of my frd told about this website, I am thinking every one will explain common things.but very very very useful

  7. Could You please explain about Folders and Covers concept with some real time scenarios….also Application restriction to versions and application locking as well Application ruleset stack

    1. It is because, thread and requestor scope data page applicable for a particular requestor. They can update the data page and it will be reflected to that user. But, Node level data pages are available for all the requestors in a particular node. So if you make it editable, it can be reflected to other users too, but comes with a huge performance issue. So pega played wisely. Node level pages are read only and cannot be updated by any user

  8. Hi Prem,

    In the Advantage of key paged access , you mentioned the advantage of it as “Avoid Multiple rules management “-> what is the meaning of it.

    1. It means a single data page can appear as a single page or multiple pages ( list type). So we don’t want to create separate data pages for single page type or list page type

  9. Hi Prem,
    I went through few of the topics and I have only word to say “Awesome”.The best part of the blog is you try to touch almost everything for each topics in a nutshell with tremendous explanation.
    One thing Prem,can we have a discussion on Obj and RDB methods.

  10. Hi Prem

    Accidentally I found your blog. When I have gone through your explanations, I feel so happy about my accidental Discovery:). It really helps a lot when we understand a concept with real time scenarios. Can we expect Knowledge share on case management.

    1. Hi Ronali,

      I’m feeling very elated to hear from you that you like it and it is very much helpful to you. 😀
      Thank you so much for your appreciation. 🙂
      Yeah, I’ll post about case management soon. Stay tuned 🙂

  11. He Prem,

    All your posts are excellent and very useful.
    Could you please let us know the techniques how to use Data Pages effectively in terms of performance level.

    Thanks
    Rajesh

    1. Thanks Rajesh.
      data pages significantly improve the preformance.
      1. You can source the data page with connectors and load the data in the clipboard. You can use that data any time.
      2. You can configure the data page to be node level, so that all requestors can access the data page. In one single hit, we can load the data in all requestors clipboard page.
      3. In activity, you can use Load-datapage along with Connect-Wait to load the data asynchronously. This improves the preformance.
      4. You can build you own refresh strategy

    1. Hi Pradeep,

      I’ll post about them soon. I’m just a little busy with works right now, but I’ll post about them soon. Stay tuned. 🙂

      Regards,
      Premkumar G

  12. Really good stuff. I have a query regarding the Edit mode of the data page. When should one opt for read only mode and when to go for editable.

    1. Data Pages
      Read only – Use read only mode, when you want to use the data page results only for display purpose. If you try to set any property on the read only data page, then you may end up with error.
      Editable – Some data page can be made editable. For example, you use a grid with data page as source. If you want to select the results, then you can use ‘pySelected’property and edit the page results ( true or false).

  13. Prem, you are awesome dude.
    I feel this blog as a your “great service” to pega learners.
    May god give you more energy and strength to keep this going.
    Please don’t stop writing and sharing. I like your simplicity in content and explanation.

    1. Dear Sharad,
      Thank you so much for your appreciation, Sharad. I’m on cloud nine reading your comments. Thank you so much for your wishes. 😀
      Don’t worry bro I’ll not stop writing. I’ll post more and more new stuffs. Stay tuned. 🙂
      Glad to hear that you like my blog. 😀

  14. Hi Anna,

    Thanks for sharing a greatfull data for us.You are really good anna i learned from two different people but no one will give this type of sitauational concepts for us.

    I already subscribed .

    Thanks,
    A Venkata Dileep.

  15. I want to thank you for ur detail information and for ur great sharing mind rare people like you do such things. ur really a ocean of good drop

    1. You are most welcome, Srinivasa. 🙂
      Thank you so much for your heartfelt appreciation. Comments like these motivate me to work hard more. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *