Please see my other blog for Oracle EBusiness Suite Posts - EBMentors

Search This Blog

Note: All the posts are based on practical approach avoiding lengthy theory. All have been tested on some development servers. Please don’t test any post on production servers until you are sure.

Saturday, December 04, 2010

Oracle GoldenGate Tutorial Part 1 - Concepts and Architecture

What is Oracle Golden Gate
Oracle GoldenGate is a comprehensive software package for enabling the replication of data in heterogeneous data environments.
The product set enables high availability solutions, real-time data integration, transactional change data capture, data replication,
transformations, and verification between operational and analytical enterprise systems.
It enables the exchange and manipulation of data at the transaction level as well as changes to DDL (data definition language2) across a variety of topologies.
With this flexibility, and the filtering, transformation, and custom processing features of
Oracle GoldenGate, you can support numerous business requirements:
● Business continuance and high availability.
● Initial load and database migration.
● Data integration.
● Decision support and data warehousing.

Oracle GoldenGate architecture

Oracle GoldenGate is composed of the following components:
● Extract Process
● Data pump Process
● Replicat Process
● Trails or extract files
● Checkpoints
● Manager Process
● Collector Process

The Extract process runs on the source system and is the extraction (capture) mechanism
of Oracle GoldenGate. You can configure Extract in one of the following ways:
● Initial loads: For initial data loads, Extract extracts a current set of data directly from
their source objects.
● Change synchronization: To keep source data synchronized with another set of data,
Extract captures changes made to data (typically transactional inserts, updates, and
deletes) after the initial synchronization has taken place. DDL changes and sequences
are also extracted, if supported for the type of database that is being used.
Data pumps
This is the optional GG process runs on the source system. Without data pump process, extract must send data to a remote trail on the target. If you use data pump then primary extract process writes to a trail on the source system. The data pump reads this trail and sends the data over the network to a remote trail on the target.
The Replicat process runs on the target system. Replicat reads extracted data changes and
DDL changes (if supported) that are specified in the Replicat configuration, and then it replicates them to the target database.
To support the continuous extraction and replication of database changes, Oracle GoldenGate stores the captured changes temporarily on disk in a series of files called a trail.
Trail files are created as needed during processing, and they are aged automatically to allow processing to continue without interruption for file maintenance. All file names in a trail begin with the same two characters, which you assign when you create the trail.
Checkpoints store the current read and write positions of a process to disk for recovery purposes. These checkpoints ensure that data changes that are marked for synchronization actually are extracted by Extract and replicated by Replicat, and they prevent redundant processing.
Manager is the control process of Oracle GoldenGate. Manager must be running on each system in the Oracle GoldenGate configuration before Extract or Replicat can be started. It performs managerial tasks like monitoring , restarting the golden gate processes , maintaining trail files etc.
Collector is a process that runs in the background on the target system. Collector receives extracted database changes that are sent across the TCP/IP network, and it writes them to a trail or extract file. Typically, Manager starts Collector automatically when a network connection is required. When Manager starts Collector, the process is known as a dynamic Collector, and Oracle GoldenGate users generally do not interact with it.

Other Concepts
Processing Group
To differentiate among multiple Extract or Replicat processes on a system, you define processing groups.
A processing group consists of a process (either Extract or Replicat), its parameter file, its checkpoint file, and any other files associated with the process. For Replicat, a group also includes a checkpoint table, if one is being used.
Commit Sequence Number (CSN)
A CSN is an identifier that Oracle GoldenGate constructs to identify a transaction for the purpose of maintaining transactional consistency and data integrity. It can be required to position Extract in the transaction log, to reposition Replicat in the trail, or for other purposes.
Each kind of database management system generates some kind of unique serial number of its own at the completion of each transaction, which uniquely identifies that transaction.  Extract writes a normalized form of the CSN to external storage such as the trail files and the checkpoint file.

Related Posts:
Oracle GoldenGate Tutorial Part 2 - Installation (Windows 2003) 
Oracle GoldenGate Tutorial Part 3 - Manager process 
Oracle GoldenGate Tutorial Part 4 - Working with OGG 
Live Reporting with OGG 
Live Reporting with Data Pump -GG 
OGG Reference - Commands and Options 
SQL Server to SQL Server Replication using GG

No comments: