In Oracle to SQL Server replication, if the Oracle source database is running on Linux then setup wizard is executed on target Windows machine where SQL Server database is running. 3rd party products such as MS SSMA or Navicat Premium (Navicat is recommended only for small tables) can be used to load data from oracle to SQL Server in order to sync source and target environment before initiating replication.
Click here for prerequisite checklist.
Setup Wizard Example
This example shows an Oracle-to-SQL Server one-way replication using the following information.
|Source database||reptest2 (Oracle)|
|Source server||dbvldemo102 (Linux)|
|Target database||w420g (SQL Server)|
|Target server||dbvisit420 (Windows 2008)|
|TNS alias for source database||reptest2|
First of all open the Dbvisit Replicate command console from the start menu and execute 'setup wizard' command.
Throughout the wizard, defaults are shown in [brackets]. Just press enter to accept them. The replication name in this example is "reptest2".
dbvrep> setup wizard This wizard configures Dbvisit Replicate to start a replication process. The setup wizard creates configuration scripts, which need to be run after the wizard ends. No changes to the databases are made before that. The progress is saved every time a list of databases, replications, etc. is shown. It will be re-read if wizard is restarted and the same DDC name and script path is selected. Run the wizard now? [yes] yes Accept end-user license agreement? (view/yes/no) [view] yes Before starting the actual configuration, some basic information is needed. The DDC name and script path determines where all files created by the wizard go (and where to reread them if wizard is rerun) and the license key determines which options are available for this configuration. (DDC_NAME) - Please enter a name for this replication (suggestion: use the name of the source database):  reptest2 (LICENSE_KEY) - Please enter your license key (or just enter "(trial)"): [(trial)] (SETUP_SCRIPT_PATH) - Please enter a directory for location of configuration scripts on this machine: [C:\Users\oracle\Documents\reptest2]
Network configuration (TNS)
Dbvisit Replicate will detect tnsnames.ora locations on your system. Choose the TNS configuration that is applicable.
Network configuration files were detected on this system in these locations: C:\app\oracle\product\11.2.0\grid\network\admin C:\app\oracle\product\11.2.0\dbhome_1\network\admin
Describing the databases
During this step information on the databases used in the replication is collected. The result is a SQL*Plus SQLCMD scripts that recreates the schema user and grants it required privileges.
Describe the first database:
Step 1 - Describe databases ======================================== The first step is to describe databases used in the replication. There are usually two of them (source and target); however, there can be just one (when source and target is the same) or more than two (one-to-many or other complex configurations.) Let's configure the database, describing it's type, connectivity, user names etc. What type of database is this? (Oracle/MySQL/MSSQL): [Oracle] Please enter database TNS alias:  reptest2 Please enter SYSDBA user name: [SYS] Please enter password for this user: [change_on_install] ****** Please enter user with DBA role: [SYSTEM] Please enter password for this user: [manager] ****** Connecting to database reptest2 as SYSTEM to query list of tablespaces and to detect ASM (by looking whether any redo logs or archived logs are stored in ASM). Enter the Dbvisit Replicate owner and apply user (this user will be created by this script): [dbvrep] Please enter password for this user: [dbvpasswd] Permanent tablespaces detected on the database: USERS. Please enter default permanent tablespace for this user: [USERS] Temporary tablespaces detected on the database: TEMP. Please enter default temporary tablespace for this user: [TEMP] Following databases are now configured: 1: Oracle reptest2, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00 Enter number of database to modify it, or "add", or "done": [add]
Adding the second database:
Let's configure the database, describing its type, connectivity, user names etc. What type of database is this? (Oracle/MySQL/MSSQL): [Oracle] MSSQL Please enter server name (as specified in "sqlcmd -S", e.g. WINSVR1\SQLEXPRESS): [localhost] dbvisit420 Please enter database ODBC DNS:  mstest Please enter user name of an administrator: [SA] dbvuser Please enter password for this user: [password] ******* Enter the user to log into apply database: [SA] dbvuser Please enter password for this user: [password] ******* Enter the database to use: [dbvrep] Enter the schema the table are created by the user specified: [dbo] Enter the database with the user data:  w420g Following databases are now configured: 1: Oracle reptest2, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00 2: MSSQL mstest, dbvuser/***, dbvuser/***, dbvuser/***, /, dbvrep/dbo, ASM:n/a, TZ: Enter number of database to modify it, or "add", or "done": [done]
During this step the source and target databases are set for each replication pair. In Oracle to SQL Server replication, Only Oracle database can be source.
Step 2 - Replication pairs ======================================== The second step is to set source and targets for each replication pair. This is usually just choosing the first database a source and the second one as target, but many more configurations are possible. Let's configure the replication pair, selecting source and target. Following databases are described: 1: reptest2 (Oracle) 2: mstest (MSSQL) (cannot be source, is not Oracle) Select source database:  Select target database:  Use fetcher to offload the mining to a different server? (yes/no) [no] (NETWORK_QUALITY) - Please specify your network type (LAN or WAN). Autoconfigures timeouts, use of compression etc. [LAN] Lock and copy the data initially one-by-one or at a single SCN? (one-by-one/single-scn/ddl-only/resetlogs) [single-scn] Following replication pairs are now configured: 1: reptest2 (Oracle) ==> mstest (MSSQL), DDL: no, fetcher: no, process suffix: (no suffix), network: LAN, prepare type: single-scn, data load: ddl_file Enter number of replication pair to modify it, or "add", or "done": [done]
Schema/Tables to be replicated
The third step is to choose the schemas and tables to be replicated.You can specify coma separated list of multiple tables or/and schema.
Step 3 - Replicated tables ======================================== The third step is to choose the schemas and tables to be replicated. If the databases are reachable, the tables are checked for existence, datatype support, etc., schemas are queried for tables. Note that all messages are merely hints/warnings and may be ignored if issues are rectified before the scripts are actually executed. Note the following assumptions are made in this wizard Î“Ã�Ã´ which can be modified by editing the resulting script: 1. All replicated tables have a primary key defined. 2. All columns of the tables and all tables of the specified schemas are replicated, and if DDL support is enabled, whenever a new column/table is added, this should be replicate as well. 3. If an apply conflict arises, the default option is to try again repeatedly, until a different option is given or the underlying issue is resolved. 4. If an apply conflict arises, no more data will be replicated until the issue is resolved or ignored. You can also specify some advanced options: 1. Rename schemas or tables. 2. Specify filtering conditions. 3. (Tables only) Configure Change Data Capture; this does not maintain a copy of the source table, but logs all operations as separate entries. This is useful for ETL or as an audit trail. This usually requires adding of new columns (timestamps, old/new values etc.) to the target table. Dbvisit Replicate can create the target table with the additional columns during setup by selecting "ddl_run" in the data copy section of Step 2. Following tables are defined for replication pairs: 1: reptest2 (Oracle) ==> mstest (MSSQL), DDL: no, suffix: (no suffix), prepare: single-scn No tables defined. Enter number of replication pair to modify it, or "done":  Please enter list of all individual tables to be replicated. Enter schema name(s) only to replicate all tables in that schema. Use comma or space to delimit the entries. Enter the tables and schemas:  avi.sample Selected tables: avi.sample Add more tables or schemas? (YES/NO) [NO] Specify rename name, filter condition, CDC/Audit/ETL for any of the specified tables (yes/no): [no] yes Rename SCHEMA name for avi.sample (empty means no rename): [w420g.dbo] Rename TABLE name for avi.sample (empty means no rename):  Filter the data to be replicated? If yes, use single condition for ALL DML operations or use CUSTOM conditions for each operation? (NO/ALL/CUSTOM) [NO] Configure change data capture for change auditing or real-time BI? (NO/YES) [NO] Following tables are defined for replication pairs: 1: reptest2 (Oracle) ==> mstest (MSSQL), DDL: no, suffix: (no suffix), prepare: single-scn avi.sample => w420g.dbo.sample Enter number of replication pair to modify it, or "done": [done]
Rename SCHEMA name for avi.sample (empty means no rename):  w420g.dbo
Here, w420g is the target SQL Server database name and dbo is schema. Please use . format in "Rename Schema"
This steps configures the replication processes for each replication. This includes setting the
- port numbers
- Notification settings and thresholds
Add process information for the MINE process:
Step 4 - Process configuration ======================================== The fourth step is to configure the replication processes for each replication. Although most options have reasonable defaults, manual input is required. Following processes are defined: 1: MINE on reptest2 Not configured. 2: APPLY on mstest Not configured. Enter number of process to modify it, or "done":  Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using fetcher): [dbvldemo102] Server type (Windows/Linux/Unix): [Linux] Enable email notifications about problems (yes/no)? [YES] no Enable SNMP traps/notifications about problems (yes/no)? [NO] Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)? [/home/oracle/reptest2] Following settings were pre-filled with defaults or your reloaded settings: ---------------------------------------- [MINE_REMOTE_INTERFACE]: Network remote interface: dbvldemo102:7901 [MINE_DATABASE]: Database TNS: reptest2 [TNS_ADMIN]: tnsnames.ora path: C:\app\oracle\product\11.2.0\dbhome_1\network\admin [MINE_USER]: Dbvisit Replicate database username: dbvrep [MINE_PASSWORD]: Dbvisit Replicate database password: ********* [MINE_PLOG]: Filemask for generated plogs: /home/oracle/reptest2/mine/%S.%E (%S is sequence, %T thread, %F original filename (stripped extension), %P process type, %N process name, %E default extension) [LOG_FILE]: General log file: /home/oracle/reptest2/log/dbvrep_%N_%D.%E [LOG_FILE_TRACE]: Error traces: /home/oracle/reptest2/log/trace/dbvrep_%N_%D_%I_%U.%E Checking that these settings are valid... Do you want change any of the settings? [no] Following processes are defined: 1: MINE on reptest2 Host: dbvldemo102, SMTP: NO, SNMP: NO 2: APPLY on mstest Not configured.
TNS_ADMIN value for MINE process is invalid because MINE process will run on Linux machine having different TNS_ADMIN than one on the current Windows machine. You can change its value here in setup wizard or by manually editing MINE DDC file later. This value must be changed later before starting MINE process.
Add process information for the APPLY process:
Enter number of process to modify it, or "done":  Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using fetcher):  dbvisit420.dbvisitdomain.dbvisit.com Server type (Windows/Linux/Unix):  Windows Enable email notifications about problems (yes/no)? [YES] no Enable SNMP traps/notifications about problems (yes/no)? [NO] Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)? [C:\Users\oracle\Documents\reptest2] Following settings were pre-filled with defaults or your reloaded settings: ---------------------------------------- [APPLY_REMOTE_INTERFACE]: Network remote interface: dbvisit420.dbvisitdomain.dbvisit.com:7902 [APPLY_DATABASE]: Database ODBC DNS: mstest [TNS_ADMIN]: tnsnames.ora path: C:\app\oracle\product\11.2.0\dbhome_1\network\admin [APPLY_USER]: Dbvisit Replicate database username: dbvuser [APPLY_PASSWORD]: Dbvisit Replicate database password: ******** [APPLY_SCHEMA]: Dbvisit Replicate database: dbvrep [APPLY_SCHEMA2]: Dbvisit Replicate schema: dbo [APPLY_STAGING_DIR]: Directory for received plogs: C:\Users\oracle\Documents\reptest2\apply [LOG_FILE]: General log file: C:\Users\oracle\Documents\reptest2\log\dbvrep_%N_%D.%E [LOG_FILE_TRACE]: Error traces: C:\Users\oracle\Documents\reptest2\log\trace\dbvrep_%N_%D_%I_%U.%E Checking that these settings are valid... Do you want change any of the settings? [no] Following processes are defined: 1: MINE on reptest2 Host: dbvldemo102, SMTP: NO, SNMP: NO 2: APPLY on mstest Host: dbvisit420.dbvisitdomain.dbvisit.com, SMTP: NO, SNMP: NO Enter number of process to modify it, or "done": [done]
All the information about the replication process has been obtained and now the replication environment can be created.
Created file C:\Users\oracle\Documents\reptest2\reptest2-APPLY.ddc. Created file C:\Users\oracle\Documents\reptest2\reptest2-MINE.ddc. Created file C:\Users\oracle\Documents\reptest2\reptest2-setup.dbvrep. Created file C:\Users\oracle\Documents\reptest2\reptest2-dbsetup_reptest2.sql. Created file C:\Users\oracle\Documents\reptest2\reptest2-dbsetup_mstest.sql. Created file C:\Users\oracle\Documents\reptest2\reptest2-grants_reptest2.sql. Created file C:\Users\oracle\Documents\reptest2\reptest2-grants_mstest.sql. Created file C:\Users\oracle\Documents\reptest2\reptest2-onetime.ddc. Created file C:\Users\oracle\Documents\reptest2\start-console.bat. Created file C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.dbvisitdomain.dbvisit.com.bat. Created file C:\Users\oracle\Documents\reptest2\reptest2-win_services-dbvisit420.dbvisitdomain.dbvisit.com.bat. Created file C:\Users\oracle\Documents\reptest2\reptest2-run-dbvldemo102.sh. Created file C:\Users\oracle\Documents\reptest2\Nextsteps.txt. Created file C:\Users\oracle\Documents\reptest2\reptest2-all.bat. =========================================================================================================================== Dbvisit Replicate wizard completed Script C:\Users\oracle\Documents\reptest2\reptest2-all.bat created. This runs all the above created scripts. Please exit out of dbvrep, review and run script as current user to setup and start Dbvisit Replicate. =========================================================================================================================== dbvrep> exit
The Dbvisit Replicate wizard has completed and has created the necessary replication environment.
Exit out of dbvrep and run -all.bat script created by the wizard
In the above example the script is called reptest2-all.bat. This script can be run at any time to reset or recreate the replication environment. The Nextsteps.txt explains the next steps to initiate the replication.
This completes the Dbvisit Replicate setup and configuration.
Before running the script, review the created scripts as indicated by the Nextsteps.txt (shown at the end of all.* script). For example:
- Review the location of dbvrep(.exe) if it differs among the nodes if RAC is used.
- Check TNS_ADMIN and ORACLE_HOME paths and make sure they are correct on all the different servers involved.
Outcome of running the scripts created by the setup wizard.
Running the reptest2-all.bat script will configure and initiate the replication. It does the following:
- Create the Dbvisit Replicate schemas in the source and target databases.
- Grant the necessary privileges to the Dbvisit Replicate schema.
- Create the Dbvisit Replicate repository in the source and target databases.
- Load the Dbvisit Replicate configuration file (DDC) into the Dbvisit Replicate repository. The DDC file is like the init.ora for Oracle and contains the settings for the replication.
- Prepare the schemas or objects for replication.
Example of running the reptest2-all.bat
C:\Users\oracle\Documents\reptest2>reptest2-all.bat Setting up Dbvisit Replicate configuration Configure database reptest2... Configure database mstest... Msg 3701, Level 11, State 1, Server DBVISIT420\SQLTEST, Line 1 Cannot drop the database 'dbvrep', because it does not exist or you do not have permission. Object grants for database reptest2... Object grants for database mstest... Setting up the configuration Initializing......done WARN-1850: No DDC DB available, dictionary table does not exist. DDC loaded from database (0 variables). Copyright (C) Dbvisit Software Limited. All rights reserved. DDC file C:\Users\oracle\Documents\reptest2\reptest2-onetime.ddc loaded. MINE: Cannot determine Dbvisit Replicate dictionary version. (no dictionary exists) APPLY: Cannot determine Dbvisit Replicate dictionary version. (no dictionary exists) dbvrep> set ON_WARNING SKIP Variable ON_WARNING set to SKIP for process *. dbvrep> set ON_ERROR EXIT Variable ON_ERROR set to EXIT for process *. dbvrep> ENGINE SWITCH_REDOLOG dbvrep> ENGINE SETUP MINE DROP DICTIONARY 0 dictionary objects dropped. dbvrep> ENGINE SETUP MINE CREATE DICTIONARY dbvrep> ENGINE SETUP MINE LOAD DICTIONARY Supplemental logging on database set. Loading dictionary table DBRSCOL$ Loading dictionary table DBRSOBJ$ Loading dictionary table DBRSTAB$ Loading dictionary table DBRSUSER$ Loading dictionary table DBRSV_$DATABASE dbvrep> ENGINE SETUP APPLY DROP DICTIONARY 0 dictionary objects dropped. dbvrep> ENGINE SETUP APPLY CREATE DICTIONARY dbvrep> ENGINE SETUP APPLY LOAD DICTIONARY dbvrep> ENGINE SETUP PAIR MINE AND APPLY ID of mine proces is 8EFF00AA-6D9B-1014-B197-EC251B966082. If not using DDC in database, set MINE_UNIQUE_ID to this value. 1 applier SCN set. dbvrep> SET APPLY.INSTANTIATE_SCN NOW Variable INSTANTIATE_SCN set to NOW for process APPLY. dbvrep> memory_set IGNORE_APPLY_DDL_DIFFERENCES YES Variable IGNORE_APPLY_DDL_DIFFERENCES set to YES for process *. dbvrep> ENGINE SUPPLEMENTAL LOGGING TABLE avi.sample ENABLE PRIMARY KEY dbvrep> ENGINE SWITCH_REDOLOG dbvrep> #single-scn instantiation: lock all tables and schemas dbvrep> ENGINE LOCK TABLES avi.sample Locking all tables. Lock done. dbvrep> #single-scn instantiation: unlock all tables and schemas, but keep the SCN dbvrep> ENGINE LOCK RELEASE LOCKS dbvrep> dbvrep> #prepare the tables (we use OFFLINE as neither MINE not APPLY is running; with OFFLINE we won't wait on network timeout) dbvrep> PREPARE OFFLINE TABLE avi.sample NODDL RENAME TO w420g.dbo.sample Table avi.sample instantiated at SCN 1413199 dbvrep> #single-scn instantiation: unlock all tables and schemas, forget the SCN (so it does not affect any further PREPARE statements) dbvrep> ENGINE LOCK CLEAR SCN dbvrep> ENGINE SWITCH_REDOLOG dbvrep> #prepare script for instantiation dbvrep> ENGINE PREPARE_DP WRITE DDL_FILE FILE C:\Users\oracle\Documents\reptest2\APPLY.sql USERID SYSTEM/odb10g@reptest2 Created DDL script C:\Users\oracle\Documents\reptest2\APPLY.sql. dbvrep> create ddcdb from ddcfile DDC loaded into database (204 variables). dbvrep> set ON_WARNING SKIP Variable ON_WARNING set to SKIP for process *. dbvrep> set ON_ERROR SKIP Variable ON_ERROR set to SKIP for process *. OK-0: Completed successfully. These steps are required after the reptest2-all.bat script runs: 1) Create the necessary directory(ies) on the servers: dbvisit420.dbvisitdomain.dbvisit.com: C:\Users\oracle\Documents\reptest2 dbvldemo102: /home/oracle/reptest2 2) Copy the DDC files to the server(s) where the processes will run: dbvisit420.dbvisitdomain.dbvisit.com: C:\Users\oracle\Documents\reptest2\reptest2-APPLY.ddc dbvldemo102: C:\Users\oracle\Documents\reptest2\reptest2-MINE.ddc 3) Review that path to dbvrep executable is correct in the run scripts: C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.dbvisitdomain.dbvisit.com.bat C:\Users\oracle\Documents\reptest2\reptest2-run-dbvldemo102.sh 4) Review and run the scripts for Windows to create services (run with Administrator privileges): C:\Users\oracle\Documents\reptest2\reptest2-win_services-dbvisit420.dbvisitdomain.dbvisit.com.bat 5) Copy the run script to the server(s) where the processes will run: C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.dbvisitdomain.dbvisit.com.bat C:\Users\oracle\Documents\reptest2\reptest2-run-dbvldemo102.sh 6) Ensure firewall is open for listen interfaces 0.0.0.0:7902, 0.0.0.0:7901 used by the processes. 7) Make sure the data on apply are in sync as of time when setup was run. Scripts for Data Pump/export/DDL were created as requested: C:\Users\oracle\Documents\reptest2\APPLY.sql Create referenced database links (if any) before running the scripts. 8) Start the replication processes on all servers: C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.dbvisitdomain.dbvisit.com.bat C:\Users\oracle\Documents\reptest2\reptest2-run-dbvldemo102.sh 9) Start the console to monitor the progress: C:\Users\oracle\Documents\reptest2\start-console.bat The above list is stored in C:\Users\oracle\Documents\reptest2\Nextsteps.txt. Press any key to continue . . .
The replication has been configured and installed. Steps required to start replication are saved in Nextsteps.txt file.
If you have empty source tables then you can execute APPLY.sql to create tables in SQL server. Otherwise you can use Navicat Premium to load data from Oracle into SQL Server.
Executing reptest2-all.bat script creates dbvrep database in SQL Server to store metadata of replication. Set correct snapshot isolation for dbvrep. Click here for more info.
Starting the replication
As indicated by the Nextsteps.txt , the replication can be started by following the steps that are listed above.
Starting the replication does the following:
- Starts the MINE and APPLY process in the background
- Creates the necessary log and PLOG directories.
- Once the APPLY and MINE process has been started, there is in INITIALIZATION process. This replicates the contents of the Dbvisit Replication repository from the source to the target database. The replication can not be started until this phase has been completed.
Edit *-Mine.DDC file on Linux and change TNS_ADMIN value before starting MINE process.
Due to *-run-.sh is created on Windows environment, Executing this script on Linux can give an error about "dbvrep" executable location.
Please use dos2unix -O command before executing the script. (or use text editor that can handle windows to linux text - notepad++ etc.)
Starting the console
The dbvrep console can be started by executing start-console.bat script to monitor the real time status of the replication.
The console can only be started on Windows machine (Target). Future release will overcome setup-wizard and console limitations.
Following is the reptest2 replication console showing current status of replication. Initialization of APPLY process may take 5 to 7 minutes
-MINE IS running. Currently at plog 425 and SCN 12377134 (09/05/2012 21:58:15). APPLY IS running. Currently at plog 425 and SCN 12377168 (09/05/2012 21:58:27). Progress of replication reptest1:MINE->APPLY: total/this execution --------------------------------------------------------------------------------------------------------------------------------- AVI.SAMPLE: 100% Mine:1/1 Unrecov:0/0 Applied:1/1 Conflicts:0/0 Last:05/09/2012 21:58:30/OK --------------------------------------------------------------------------------------------------------------------------------- 1 tables listed.
Scripts generated by setup wizard contains plain text passwords. You can delete these scripts once replication is configured and running.