z/OS: Problem Determination
- z/OS: Problem Determination
- Figures
- Tables
- About this book
- Who should read this book
- What we need to know to understand this book
- How to use this book
- Summary of changes
- Changes for this edition (GC34-6600-00)
- Establishing the cause of the problem
- Introduction to problem determination
- How this book can help you
- How this book is organized
- Preliminary checks
- Has WebSphere MQ for z/OS run successfully before?
- Are there any error messages or return codes that explain the problem?
- Can you reproduce the problem?
- Have any changes been made since the last successful run?
- Has the application run successfully before?
- Does the problem affect specific parts of the network?
- Does the problem occur at specific times of the day?
- Does the problem affect all users of the application?
- Does the problem occur with all z/OS, CICS, or IMS systems?
- Is the problem intermittent?
- Have you applied any APARs or PTFs?
- Do you have a program error?
- What to do next
- Examining the problem in greater depth
- Have you obtained some incorrect output?
- Have you received an unexpected error message?
- Has there been an abend?
- Common causes of abends
- Address space dumps and transaction dumps
- Abnormal program termination
- Have you failed to receive a response from an MQSC command?
- Is there a problem with the WebSphere MQ queues?
- Are some of your queues working?
- Are the correct queues defined?
- Does the problem affect only remote or cluster queues?
- Does the problem affect only shared queues?
- Is your application or WebSphere MQ for z/OS running slowly?
- Has your application or WebSphere MQ for z/OS stopped processing work?
- Check for error messages
- No error messages issued
- Has the queue manager or channel initiator terminated abnormally?
- WebSphere MQ for z/OS might still be running
- Where to look next
- Dealing with the problem
- Dealing with program abends
- Batch abends
- CICS transaction abends
- IMS transaction abends
- Dealing with waits and loops
- Distinguishing between waits and loops
- Waits
- Loops
- Symptoms of waits and loops
- Dealing with waits
- Is a batch or TSO program waiting?
- Is a CICS transaction waiting?
- Is DB2 waiting?
- Is RRS active?
- Is WebSphere MQ for z/OS waiting?
- Dealing with loops
- Is a batch application looping?
- Is a batch job producing a large amount of output?
- Does a CICS region show a lot of CPU activity?
- Does an IMS region show a lot of CPU activity?
- Is the queue manager showing a lot of CPU activity?
- Is a queue, page set, or Coupling Facility structure filling up unexpectedly?
- Are a task, and WebSphere MQ for z/OS, showing a lot of CPU activity?
- Dealing with performance problems
- z/OS system considerations
- WebSphere MQ for z/OS considerations
- Log buffer pools
- Buffer pool size
- Distribution of data sets on available DASD
- Distribution of queues on page sets
- Distribution of queues on Coupling Facility structures
- Limitation of concurrent threads
- Using the WebSphere MQ trace for administration
- CICS constraints
- Application design considerations
- Effect of message length
- Effect of message persistence
- Searching for a particular message
- Queues that contain messages of different lengths
- Frequency of syncpoints
- Advantages of the MQPUT1 call
- Dealing with incorrect output
- Messages do not appear when expected
- Problems with missing messages when using distributed queuing
- Problems with getting messages when using message grouping
- Finding messages sent to a cluster queue
- Finding messages sent to the WebSphere MQ-IMS bridge
- Messages contain unexpected or corrupted information
- Diagnostic aids and techniques
- Diagnostic aids
- WebSphere MQ for z/OS recovery actions
- Program errors
- WebSphere MQ for z/OS abends
- WebSphere MQ for z/OS diagnostic information
- z/OS abends
- CICS abends
- IMS abends
- Diagnostic information produced
- Error messages
- Dumps
- Console logs and job output
- Symptom strings
- Queue information
- Other sources of information
- Your own documentation
- Manuals for the products you are using
- Source listings and link-edit maps
- Change log
- System configuration charts
- Information from the DISPLAY CONN command
- Diagnostic aids for CICS
- Diagnostic aids for IMS
- Diagnostic aids for DB2
- WebSphere MQ dumps
- How to use dumps for problem determination
- Getting a dump
- Using the z/OS DUMP command
- Processing a dump
- Using the WebSphere MQ for z/OS dump display panels
- Using line mode IPCS
- Using IPCS in batch
- Analyzing the dump
- Dump title variation with PSW and ASID
- SYSUDUMP information
- Snap dumps
- SYS1.LOGREC information
- Finding the applicable SYS1.LOGREC information
- When SVC dumps are not produced
- Suppressing WebSphere MQ for z/OS dumps using z/OS DAE
- Using trace for problem determination
- The user parameter and IBM internal traces
- Starting the trace
- Formatting the information
- If trace data is not produced
- Interpreting the information
- The Log and Trace Analyzer tool
- Examples of trace output
- Other types of trace
- The channel initiator trace
- The line trace
- The CICS adapter trace
- System SSL trace
- z/OS traces
- Finding solutions to similar problems
- Searching the IBM database
- Search argument process
- Techniques for varying the search
- The keyword format
- Free format
- Structured database (SDB) format
- Building a keyword string
- Component-identifier keyword
- Release-level keyword
- Type-of-failure keyword
- Abend keyword
- IEA911E message
- CSQV086E message
- CSECT keyword
- Load module modifier keyword
- Recovery routine modifier keyword
- Wait and loop keywords
- Message keyword
- Procedure for WebSphere MQ for z/OS messages
- Performance keyword
- Documentation keyword
- Incorrect output keyword
- Working with IBM to solve your problem
- Reporting a problem to the IBM software support group
- When to contact the support center
- Dealing with the support center
- What the support center needs to know
- What happens next
- Collecting documentation for the problem
- General documentation needed for all problems with WebSphere MQ for z/OS
- Sending the documentation to the change team
- Resolving a problem
- The APAR process
- Applying the fix
- The APAR becomes a PTF
- Appendixes
- Appendix A. SDB format symptom-to-keyword cross reference
- Appendix B. WebSphere MQ component and resource manager identifiers
- Appendix C. CICS adapter trace entries
- Appendix D. Examples of CEDF output
- MQOPEN
- MQCLOSE
- MQPUT
- MQPUT1
- MQGET
- MQINQ
- MQSET
- Appendix E. Notices
- Programming interface information
- Trademarks
- Index