OPE User Guide

Data Transfer

ObjectStore includes three main processes communicating with each other to manage your data:

For ObjectStore Single these three main "processes" are threads within the application process.

The following topics cover aspects of data transfer and explain how these processes work together.

Handling Data Fetches

When the ObjectStore Server sends data to an ObjectStore application, the data is placed in the application's ObjectStore cache. This is a region of memory that is controlled by the ObjectStore client linked to the application. The application has direct access to the cache. If sessions are in use, each session has its own cache. In this case, whenever a client's cache or an application's cache is mentioned in this document, it means a session's cache.

Persistent data fetched from the Server is kept in the client's cache. All persistent data must be placed in the cache before the application can access it. The cache size is determined at ObjectStore initialization in one of the following ways:

The Show details dialog of the File menu on the TimeLine diagram shows you the cache size used by the application. If the client must bring more persistent data into the cache, and no space is available, the client performs cache replacement to make room for this data.

Many aspects of data throughput and concurrent data access are determined by the state of the cache at any point during an application's execution. To understand how the cache behaves, how to control it, and how to measure its efficiency, are important aspects of ObjectStore's performance tuning.

This section describes how to use OPE to determine if an application uses its cache efficiently. For more information about cache operations refer to ObjectStore Performance, which discusses in detail the behavior of the ObjectStore cache.

How can you determine if data is transferred into the application efficiently?

OPE refers to the category containing information about data transfer from the Server to the client as fetch. OPE shows you

Shown in the ElapsedTime diagram or in the Client Counters/Ratios diagram for a single transaction or an arbitrary time range

Shown in the Storage Counters/Ratios diagram for a single transaction or an arbitrary time range

Shown in Transaction summary of the File menu on the TimeLine diagram for the entire trace

How long does it take ObjectStore to fetch the data?

From the client's perspective, a combination of the following operations is necessary to fetch data from a Server:

  1. OPE records t1, which is the time stamp for beginning fetch.


  2. Client sends data requests (for example, one or more pages or segments) to the Server.


  3. Server reads data from disk or cache (unless blocked by other clients).


  4. Server sends data to client cache.


  5. OPE records t2, which is the time stamp for finished fetch.


OPE sums the duration of these operations (t1 till t2) when reporting the duration of a single fetch event. OPE reports only the time that is spent for fetch activity. The actions that the Server takes to fulfill the request are not shown.

This means that, from OPE's perspective, a slow fetch might depend on many factors, such as network load, a busy Server process, or conflicts due to concurrent data access. During this time, the client is blocked and waits for the request to be fulfilled. (For details on concurrent data access see Concurrent Data Access.)

The best way to start an examination of fetch cost is to check the fetch costs for the entire application trace. To do this, look at the ElapsedTime diagram. It shows you the absolute time spent in fetch and a percentage of the selected diagram interval.

When you place the mouse pointer on the fetch segment of the diagram, the status line shows you percentage and duration of the fetch. By doing this, you get a feeling for the portion of time that the application spends to fetch its data.

You can examine these costs further when you look at the ElapsedTime diagram on a transaction-by-transaction basis or for arbitrary time intervals. By exploring the trace in this way, you can identify the transactions in which the time spent for fetch seems to be disproportionately long.

How much data is fetched at a time?

With OPE you can determine the amount of data that is fetched during each fetch operation. This quantity is called the fetch quantum. The application determines the fetch quantity, which can be one of the following:

Only the page needed is transferred.

The entire cluster containing the page needed is transferred (this is the default).

A stream of pages located near to the page needed is transferred.

OPE shows you this information as the counter pages fetched. In the Client Counters/Ratios diagram you see the counters for transactions; in the Storage Counters/Ratios diagram you see the counters for databases, segments, clusters, and pages.

The application controls the fetch quantum at run time on a global, database, segment, or cluster basis. To decide how much data should be fetched at a certain point in the application's execution is important for the tuning process. It requires understanding the application's pattern of access to the respective data.

What is the application's pattern of access to the data fetched?

The duration of time spent in a fetch and the fetch quantum per fetch is only part of the picture. It is also important to understand how the application accesses the data. You need to know the

When you understand these parameters, you can create a model for the application's pattern of access. Also, you can change the application's performance by changing its fetch policy and the distribution of data amongst clusters.

OPE provides you with relevant counter information in the Client Counters/Ratios diagram (as a function of time) and in the Storage Counters/Ratios diagram (as a function of database location).

First, look at the Client Counters/Ratios diagram. Zoom out to see the entire application trace and select any of the pages fetched counters. From the peaks you can determine transactions that show a lot of activity.

Then, look at the Storage Counters/Ratios diagram to determine which parts of the database your application accesses during these transactions. By doing this, you can isolate segments, clusters and pages with a lot of activity.

When you identify the clusters that contribute to the total fetch time, analyze the order in which the client application fetches the data. This helps you implement the appropriate fetch policies.

Determine the access pattern for each cluster. If the cluster is fetched

Determine the order in which the pages are fetched and the amount of fetched pages compared to the size of the segment.

Check whether the application accesses the entire fetched cluster during the time interval that you investigate.

Check whether the number of pages that are streamed is appropriate.

If the application accesses many pages scattered over many clusters and fetches only a few pages per cluster, consider reclustering the data. The application then accesses fewer pages in fewer clusters. For more details refer to ObjectStore Performance.

Then look at the PageState diagram and do the following for each cluster:

When the application changes the fetch policy during execution, OPE shows this as an environment event. To get this information, select Environment in the Select Events dialog of the TimeLine diagram.

Is the right amount of data transferred?

If the client faults on a page, and a fetch policy other than page at a time is in effect, so-called free rider pages are fetched from the Server.

There is little difference in cost between fetching one page and fetching two or three pages. If you know that the application needs many pages that are located near each other in the database, it might be advantageous to fetch them all at once.

This analysis might lead you to the decision that the entire cluster is to be fetched whenever the application faults on a single page in that cluster rather than fetching the contents of the cluster in one- or two-page increments. However, if the application never uses free rider pages, it pays the cost for fetching without getting any benefit.

The goal is to match the application's pattern of access with the pattern of fetches as closely as possible. At the same time, concurrent data access should be optimized.

OPE helps you investigate how many free rider pages are accessed by the application over a specified time interval. If you see that a large number of these pages are not used, you might change the application's fetch quantum and clustering to improve performance.

The client counter pages fetched free rider shows you the total number of free rider pages. If you see that many of these pages are used, check the storage counter to get a breakdown of these pages according to storage location on database, segment or cluster level.

If a particular cluster has many free rider pages, expand the storage object to the page level. Add the counters for pages locked or pages upgraded to see if the application accesses a large number of free rider pages. If this is the case, the application's pattern of access might match the pattern of free rider fetches.

The PageState diagram also shows you when a free rider page is fetched and accessed. An access results in a lock, so you should check if the free rider page is upgraded to at least ExLx.

Note: Using fetch policy cluster and any cluster, segment, or database locking option might prevent you from finding unused data. As these locking options lock the free rider pages for read, you cannot see if your application really accesses the page.

Handling Data Commits

When an application ends a transaction normally by calling os_transaction::commit or by executing the OS_END_TXN macro, the ObjectStore client starts the process of committing data that the application modifies during the transaction.

What happens during commit?

During commit, the client transfers modified pages to the respective Servers that fetched them for the client. The client follows a protocol called two-phase commit, if more than one server is involved in the commit.

Unless the transaction is aborted, all pages locked for write during this transaction are committed to their respective databases. The client is blocked and must wait until the data sent to the Server is written to disk.

The client can perform other tasks during this process, such as relocating out a subset of pages. (For information on outbound relocation, refer to Relocation.)

In certain cases, the client might transfer pages to the Server before a commit is requested, as a result of cache replacement. To obtain peak commit performance, analyze the amount of data involved in this process. OPE displays information that helps you with this analysis.

How much time is spent in commit?

In the TimeLine diagram OPE shows you the commit portion of an update transaction as the thicker part of the horizontal transaction line segment. OPE does not report a commit duration for read-only transactions.

On the TimeLine, look for transactions that have long commit durations. Double-click on the transaction's segment on the TimeLine to get detailed information on a transaction, such as the duration of its commit.

By looking for a lot of small, adjacent transactions you can decide if you should merge these transactions into a single transaction. Thereby, you might eliminate some commit overhead.

If the gaps between the transactions are short, and the data that is accessed remains the same, you might not have to commit a lot of small pieces. For example, you can set the transaction boundaries in a for/while-loop outside the loop. Note that this can also affect concurrent data access.

You can also use the ElapsedTime diagram to get a graphic view of the actual and relative amounts of time spent in commit. You see this information for the transaction that is currently selected in the TimeLine diagram.

Use Transaction summary to examine commit durations for all transactions in the trace so that you can identify the ones that need attention.

How much data is committed?

The client counter pages committed reports the number of pages that are sent to the Server during commit. pages committed differs from pages locked. Pages locked also includes pages locked for read, which do not have to be returned to the Server.

If there is no cache replacement activity in the transaction, the counter pages committed represents the number of pages that are locked for write during the transaction. The counter does not include modified pages that are evicted prior to commit.

The strategy for ensuring that the commit takes as little time as possible is to reduce the amount of data that must be transferred. This means reducing the number of pages that are modified. It also has a positive effect on concurrent data access and overall system throughput.

Examine pages committed in the Client Counters/Ratios diagram to find transactions for which this counter is high. Use the Storage Counters/Ratios diagram to determine which databases and segments contribute most to that number. Identify the segments with large numbers of committed pages to validate that the level of activity makes sense for the transaction.

Ensure, for example, that lock options and acquire_lock() are only used when necessary. You might also minimize the number of pages that are modified by reclustering data.

Managing Server Load

The Server is a process that controls access to ObjectStore databases on a host. The Server also manages pages of data on behalf of clients running applications.

Is your Server overloaded or underloaded? Is the network saturated?

Maintaining peak data throughput on the Server is essential, especially if many clients are involved. Any disk activity on the Server, such as data retrieval for other clients' fetch requests, might affect the commit performance of a particular client.

For example, your application's transactions commit only a few pages although the commit durations seem to be high. You should then also consider network traffic and Server disk activity.

Use the OPE Server Monitor to verify the Server activity level by checking, for example, client messages, log propagation, and KB read or written.

If you cannot monitor the Server while the application trace is created, use the ObjectStore dbutil API to record Server measurements as generic events (os_ope_generic_events). The values passed to raise (os_unsigned_int32,...) are shown as a client counter graph in the TimeLine diagram. For detailed information on generic events refer to "Defining a generic event".

Managing Cache Size

When you try to determine the optimum cache size and caching algorithms for your application, consider data access patterns and the amount of data that is accessed. The goal is to minimize the number of times the client must swap pages out of the cache and send them back to the Server. If sessions are in use, each session has its own cache. In this case, whenever a client's cache or an application's cache is mentioned in this document, it means a session's cache.

If the application's persistent working set exceeds the size of the cache, the client determines a candidate set of pages that are used less frequently. These pages are removed to make room for pages that are used more frequently. This process is called cache replacement or eviction.

You can use OPE to determine if the size of the cache is optimal for an application. The goal is to ensure that especially frequently used data is in the cache (warm) when the application needs it. You can control which data is in the cache and how long it remains there.

How is cache replacement done for differently locked pages?

During cache replacement, the client treats modified pages differently from pages locked for read:

Note: Cache replacement is different from and unrelated to callbacks due to concurrent data access. Cache replacement does not affect locking.

What is the difference between operating system paging and cache replacement?

The client makes persistent data available by mapping it into virtual memory. The operating system then pages the persistent data into physical memory where your application accesses it. To make room in physical memory, the operating system might have to page out other pages to the machine's swap space. This data can be stack data, code pages, or other pages containing persistent data.

Cache replacement is similar to paging in that way that the client swaps out persistent pages from the cache. However, operating system paging is less expensive than cache replacement because the pages paged out are stored locally in the machine's swap space.

In the Show details dialog of the TimeLine diagram, OPE shows you the number of operating system page faults that occur during application execution. You see also other related information.

What happens when the cache is larger than physical memory?

The operating system paging might increase. If your application has a large working set, this might be the best mode of operation for your application because operating system paging is less expensive than cache replacement.

What happens when the cache is smaller than physical memory?

The operating system paging might decrease. However, if your application's working set is larger than the cache, cache replacement occurs. This might not be desirable for performance reasons.

If the application creates a lot of new pages, it might be advantageous to maintain a cache that is smaller than physical memory. In this case, the newly created pages are evicted to the Server continuously during the transaction. They do not have to be returned for commit.

How much time is spent to evict data?

In the ElapsedTime diagram, the evict category represents the time spent in cache replacement for the following activities:

The ElapsedTime diagram might show you, for example, that the LRU algorithm takes longer as the size of the cache increases.

Note: During cache replacement, the client might fetch additional pages to perform, for example, outbound relocation. This fetch time is shown in the Fetch category of the ElapsedTime diagram. The page fetches are shown in the PageState diagram.

How much data is evicted?

Use the client counter pages evicted to see the total number of pages that are removed from the cache. You can display this counter in the Storage Counters/Ratios diagram to see which parts of the databases are involved.

pages evicted is the sum of the counters pages flushed (not locked or locked for read) and pages returned dirty (modified pages).

Which data are reaccessed after eviction?

The application might reaccess an evicted page before the transaction ends. If this occurs frequently, the application might have costly transfers to and from the Server each time.

For a particular time interval, use the storage counter pages refetched. This counter shows you the number of pages that are refetched after they are transferred to the client cache.

The PageState diagram shows you the state transitions for individual pages when they are evicted. You see an evict event (represented by a filled circle) on the corresponding page bar. Possible state transitions for eviction are

EWLW --> NE Page returned dirty
ExLR --> NE Page flushed from cache; Server has to keep the lock info
ER --> NE Page flushed

For further information expand all storage objects in the PageState diagram to the page level. Do this for a longer time interval or for the complete application trace.

To do this, choose Select all and Expand selected from the Edit menu until you have expanded the storage objects to the page level. Select the tiny bar size from the Options menu, which shows you one pixel per page.

You see

These pages have many NE and not-NE states. Also, fetch and evict are the dominant categories in the ElapsedTime diagram for that time period.

How can the application control cache replacement directly?

You can use the following ObjectStore functions to specify pages that the client should use as candidates for cache replacement:

Select Miscellaneous from the Select events dialog of the TimeLine diagram. OPE then displays symbols that represent the calls of these functions by the application to control cache replacement.

How can I see whether the cache is warm or cold?

OPE shows you the ratios cache hit rate and pages refetched that are computed based on client counters. These ratios are available in the TimeLine diagram and in the Storage Counters/Ratios diagram.

Cache hit rate is the number of accesses to pages that are fetched from the Server (cold) compared to the number of accesses to pages that are already in the cache (warm). A cache hit rate of 1.0 (100%) is optimal. Each time your application accesses data, it finds the data in the cache.

Pages refetched is defined as the percentage of pages that must be refetched after eviction. A page refetch rate of 0.0 (0%) is optimal because the application does not need a page after it is evicted.

If the refetch rate for an application task is high, the client might spend a lot of its time in cache replacement and refetching. You might increase the size of the cache to trade off this activity with operating system paging, which is less expensive.

How can you optimize cache replacement over the entire application run?

It is not sufficient to optimize only a single transaction. You must also analyze the previous and the succeeding transactions to find out which data is reused in several transactions, or which data is used only once (the locality of reference of the application).

Select Transaction summary from the File menu of the TimeLine diagram to get a summary of relevant counters, such as pages evicted, for the entire trace. The summary is written to an ASCII file, which can be plotted in a graphic tool or a spreadsheet. Look for peaks in the counters. Check if there are sequences of transactions that have a higher cache replacement activity.

Use the PageState diagram to identify pages or segments that are

This helps you decide how to change the application so that

It also helps you decide how to recluster data to reduce overall activity.


support@objectstore.net
Copyright © 2004 Progress Software Corporation. All rights reserved.