Pythian Blog: Technical Track

Using rsync to Copy GoldenGate Trails from DBFS to ACFS

by Michael Dinh on Jul 25, 2019 12:00:00 AM

I am currently working on project to migrate GoldenGate trail from Oracle Database File System (DBFS) to Oracle Automatic Storage Management Cluster File System (ACFS). The migration is performed in two phases. This is the first phase, in which I create ACFS and synchronize DBFS to ACFS. The rationale is to test if ACFS can sustain IO before actual migration and to ensure there are no issues with ACFS configuration. Environment: [code] GoldenGate trail from DBFS resides at /ggdata01/FS1 (source) GoldenGate trail from ACFS resides at /ggdata02/FS1 (target) [/code] Currently, rsync is being used with the following parameters: [code] -v: verbose -r: recursive -p: preserve permissions -o: preserve owner -g: preserve group -t: preserve modifications time --delete-after [/code] Syntax to copy all directories and files from /ggdata01/FS1 to /ggdata02/FS1. Any files deleted from /ggdata01/FS1 will be deleted at /ggdata02/FS1 after rsync has completed. [code] + /bin/rsync -vrpogt --delete-after /ggdata01/FS1/ /ggdata02/FS1 dirdat/ dirdat/aa000302676 ..... dirdat/aa000302713 deleting dirdat/aa000301444 ..... deleting dirdat/aa000301413 sent 7,603,242,915 bytes received 908 bytes 134,570,687.13 bytes/sec total size is 259,700,906,128 speedup is 34.16 real 0m57.447s user 0m39.999s sys 0m22.851s [/code] Check for differences between two directories. File cpe is expected to be different as this is continuously changed at source while GoldenGate processes are running. [code] + /bin/diff -rq /ggdata01/FS1 /ggdata02/FS1 Files /ggdata01/FS1/dirchk/E_DB.cpe and /ggdata02/FS1/dirchk/E_DB.cpe differ ..... Files /ggdata01/FS1/dirchk/P_TARGET.cpe and /ggdata02/FS1/dirchk/P_TARGET.cpe differ real 86m24.642s user 2m28.046s sys 21m42.619s [/code] Why did diff take so long and what was causing the delay? As it turns out, there are 1277 files from dirdat directory to compare between source and target. [code] + ls /ggdata01/FS1/dirdat/ + wc -l 1277 [/code] Why are there so many files? Currently, the trail size is 200M which is too small; hence, many files are created. If the trail size is increased to 500M then there will be only 511 trail files (1277*200/500). Also, there is a gap from source ih000302802 vs target ih000302713 as more trails were generated while diff was running for 86m. [code] + ls -alrt /ggdata01/FS1/dirdat/ + tail -rw-r----- 1 ggsuser oinstall 199999648 Jul 18 11:19 aa000302794 ..... -rw-r----- 1 ggsuser oinstall 199998923 Jul 18 11:26 aa000302801 -rw-r----- 1 ggsuser oinstall 97219382 Jul 18 11:27 aa000302802 + ls -alrt /ggdata02/FS1/dirdat/ + tail -rw-r----- 1 ggsuser oinstall 199999775 Jul 18 09:51 aa000302705 ..... -rw-r----- 1 ggsuser oinstall 199999891 Jul 18 09:58 aa000302712 -rw-r----- 1 ggsuser oinstall 199998069 Jul 18 10:00 aa000302713 [/code] Due to the duration for rsync to complete, it is scheduled to run every two hours at even hours, i.e. 00:00 2:00, 04:00, etc... [code] # script to sync dbfs to acfs for goldengate - dinh - July 17, 2019 0 */2 * * * /home/oracle/working/dinh/acfs_ggdata02_rsync.sh > /tmp/rsync_acfs_ggdata01_to_ggdata02.log 2>&1 [/code] Email log as attachment: [code] + mailx -s 'SUCCESS: rsync_acfs_ggdata01_to_ggdata02.log' -a /tmp/rsync_acfs_ggdata01_to_ggdata02.log me@mail.com [/code] In conclusion, rsync is a very good feature to use for synchronization and it was used for DBFS to ACFS migration.

No Comments Yet

Let us know what you think

See all

See all

Subscribe by email
Follow us
Love Your Data
Pythian Logo

© 2024 All rights reserved.