• Users are complaining about slowness in production. What will be the first thing you should do?
Server slowness
How about checking the proper node classification is in place? Not sure if this is the first thing to do but improper node classifications might cause slowness..
Other suggestions are welcome..
Sudipt One more suggestion is to Check system logs for any exceptions or Failures.
- Edited
Sudipt
Here are a few items that you need to verify before starting to fine-tune your codebase,
Infrastructure Guidelines:
- Check your production sizing document and confirm if the number of Nodes and DB sizing fits your requirement. If you don't have a sizing document, please prepare one and act accordingly.
- Check your available Web Nodes Memory usage. It should typically be less than 70%.
- Make sure no Agents run on Web Nodes.
- Check your Production environment Rule Cache size (the default is 20,000). If required, increase this based on your need.
- Make sure your database has at least 40% of free space. Get the list of top 20 space-consuming tables and work on an optimization plan.
Deployment Guidelines:
- Please run static assembler for all your applications once you deploy the code the Production.
- Kill all the Idle DB connections & DB queries more frequently.
General Guidelines:
- Make sure your requestor sessions are cleaned up in the case of Browser/Tab/Window close (Improper log-off).
- In case of Mash-ups please ensure you have a proper log-off mechanism for requestor clean-up.
- Analyze the Clipboard of the requestors who reports slowness. Pega recommended size of the clipboard is less than 7MB.
- Make sure you have vacuum enabled for the tables which have frequent updates
Code Guidelines:
- Run PAL to narrow down the root cause and work on the optimization process.
Run through the above guideline to narrow down your issue. Let us know once you have some findings. We can work together to have a plan to make your environment stable.
1, How to check web nodes memory usage?
2, How to check production environment rule cache? (Is it using pega API?)
- Make sure your database has at least 40% of free space. Get the list of top 20 space-consuming tables and work on an optimization plan. How to check database free space?
- How to kill idle DB connections and DB queries?
- What does static assembler do?
a month later
AnoopKrishna
Set health alerts in monitoring tools like PDC or AES, or solarwinds or netscalergateway you can explore them for deep understanding
How to kill idle DB connections and DB queries?
Once you set the alerts in monitoring tools a notification will be sent to the interested party ( pega team) like pega 005 or 29 alert, take logs and escalte to DB team to kill the long running query. If you have the access you can do it but it wont for sure.
What does static assembler do? -- > The Static Assembler analyzes the rule in the specific application and in the built-on application or applications and assembles any rules that have not yet been assembled. Processing can take from one to several hours. ( you can snippet it for deep analysis)
sam027 Thank you so much Sam. I thought these processes has to be done in server level files.
- Edited
@AnoopKrishna
Here is some additional information on top of @sam027 response,
1, How to check web nodes memory usage? -
In case of Cloud this can be verified using PDC & in case of on-premise this can be verified using AES. But in either of the scenario we can examine the detailed requestor count and memory usage of web nodes from Admin studio.
https://onestoppega.com/system-administration/requestor-types-in-pega/
2, How to check production environment rule cache? (Is it using pega API?) -
This is a setting in prconfig.xml file. which can be adjusted using the DSS prconfig/cache/alias/instancecountlimit/default. A restart is required post the DSS creation/update.
https://onestoppega.com/system-administration/dynamic-system-setting-system-setting/
3, Make sure your database has at least 40% of free space. Get the list of top 20 space-consuming tables and work on an optimization plan. How to check database free space?
This can be verified if you have Database access or can be verified using Pega support ticket. Vacuum process helps you maintain a healthy database & optimizes your query execution plan.
https://onestoppega.com/performance/how-to-optimize-query-execution-plan/
4, How to kill idle DB connections and DB queries?
Idle connections in the database can be terminated using the below script. We can also control the connection pooling of each databases at the database level.
SELECT pg_terminate_backend(pid) FROM pg_stat_activity
WHERE datname = 'databasename'
AND pid <> pg_backend_pid()
AND state in ('idle');
5, What does static assembler do?
Static assembler helps you set the rule cache of the specific application. Static assembler is required when you have a major deployment in production or if your production has very frequent deployments. In other cases, it's not really needed.
Karthick Thank you so much Karthick.::