This article describes basic network troubleshooting to resolve most common network problems for using Dbvisit Standby product.
Problem Details
Problem Statement
Troubleshooting Dbvisit Standby v8 & v9 Network Components
Applicable Error Code
N/A
Affected Versions
v8, v9
Affected Platforms
Linux, Windows
Description
In order for Dbvisit standby solution to work, you need to ensure, that network communication is possible across your standby landscape. This article focuses on diagnosing whether the network communication is possible. It does not cover troubleshooting of network quality / bandwidth problems.
Schema for network communication is show in picture below. Dbvserver can be placed on 3rd server, or be located on your standby server.
As visible in the picture, following communication via ports has to be possible (allowed by firewall):
- From administrator / user workstation => dbvserver via TCP port 4433
- From dbvserver => Primary and Standby site via port TCP 7891
- From Primary => standby via port TCP 7890
- From Standby => Primary via port TCP 7890
- From Primary => dbvserver via TCP port 4433
- From Standby => dbvserver via TCP port 4433
- If you use SSH connection instead of dbvnet, you would obviously need port 22 open in your firewall
Host name resolution has to work on all servers and also be consistent on all servers.
- For example, if standby server is named stdby1, name DNS name resolution should alway point to same IP address no matter from which server we try
By default, all dbvisit components are listening on all NICs present on the server. You should make sure, that they listen only on interfaces which are used in your Dbvisit Standby solution. For example it makes no sense for dbvagent to listen on interface for storage. Or sometimes you server has DNS names which resolves to IP address you do not want to use for Dbvisit Standby solution.
- Modify configuration files of dbvagent, dbvnet and dbvserver and change listen_address to some valid IP address / hostname.
There are 4 main points for troubleshooting your network problems:
- Check if the process is listening and on which IP it is listening
- Test, if the firewall is not preventing the connection
- Verify your hostname > IP resolution
- Diagnose, if dbvisit standby configuration files have correct parameters
Steps Performed
1) You need to make sure, that all the necessary components are running and listening on appropriate ports
LINUX:
netstat -ano | grep '4433\|7890\|7891'
In this example, everything is running fine, notice that dbvserver is listening on all interfaces, while dbvagent and dbvnet are listening on specific IP
WINDOWS:
netstat -ano | findstr /i ":4433 :7890 :7891"
Do this test on all servers to see, if everything is correct. If nothing is listening on these ports, then most probably the specific component is down and needs to be started / troubleshooted further.
2) Test if ports are allowed by firewall
For testing this fully, telnet utility is required. Unfortunately sometimes it is not installed on your server or not even allowed by your security policy. You can try to install it via yum or apt-get utilities in LINUX and via "Programs and Features" => "Turn Windows Features On or Off" in WINDOWS.
If you cannot have telnet for some reason, then you can at least perform check with dbvnet utility as specified here:
v8
v9
The testing via telnet itself is done by command:
telnet <target server> <port>
These are possible outcomes:
LINUX:
a) Communication is allowed by firewall, but there is no process listening at the target server (you immediately get back the response):
b) Communication is blocked by firewall (timeout):
c) Communication is blocked by firewall:
d) Communication is successful:
WINDOWS:
a) Communication is not allowed by firewall or there is no process listening:
NOTE: Unlike in linux, there is same message for intervening firewall / no process listening issue. If the time taken since the command is issued until the error message is displayed is greater than 10 seconds, then most probably the firewall is intervening. If the error is displayed more or less instantaneous, then it is likely, that there is no process listening.
b) If communication is allowed and something is listening on specific port, then communication is successful and only window with blinking cursor is shown:
3) Test and correct hostnames DNS name resolution
In your Dbvisit Standby landscape you usually use your hostnames (for example when installing or when creating standby database). These hostnames are stored in various configuration files. It is very important, that these hostnames are always resolved to same IP addresses across you standby landscape. You can verify this very easily by running ping command. Command is the same on windows and linux. Do not worry, if the server is reported unreachable - this is expected in many environments. Main thing is, that the hostname is correctly translated to IP address. Following examples show successful name resolution:
Following examples show unsuccessful hostname resolution:
NOTE: you should try the ping command on ALL hosts which are part of your standby landscape.
If your hostnames are not resolved correctly, it could mean, that you do not have correctly set DNS server, or that DNS server is not available. In this case, one option is to use IP addresses.
We can also make use of "local" hostnames resolution on specific server. This can be achieved by editing file /etc/hosts in LINUX and c:\Windows\System32\drivers\etc\hosts in WINDOWS. In order to add your host / IP resolution simply add line with contents <IP> <hostaname> like in this example:
192.168.4.97 primarydb 192.168.4.98 standbydb
On Windows you might need elevated privileges to edit the hosts file. Immediately after you save the file, you should be able to see, that IP resolution works for your hostnames:
Be aware - by editing your hosts file, you are bypassing your DNS (the name resolution priority could be changed, so as hosts file is not the primary method, but this is not in the scope of this article). So you are then responsible for maintenance of such hosts file.
4) Check hostnames used in your dbvsit standby installation
You should check your configuration files, whether they contain correct hostnames. The configuration files and parameters are:
Standby v8:
File | Parameter Name | Description | File Location | File Name |
DDC FILE | SOURCE | Primary server name with primary database | $DBVISIT_BASE/standby/conf | dbv_<DDC>.env |
DDC FILE | DESTINATION | Standby server name with standby database | $DBVISIT_BASE/standby/conf | dbv_<DDC>.env |
DBVNET CONF | listener_address | Local listening address (usually primary hostname) | $DBVISIT_BASE/dbvnet/conf | dbvnetd.conf |
DBVNET CONF | remote_address | Remote address (usually standby hostname) | $DBVISIT_BASE/dbvnet/conf | dbvnetd.conf |
DBVAGENT CONF | listener_address | Local listening address (usually primary hostname) | $DBVISIT_BASE/dbvagent/conf | dbvagent.conf |
DBVSERVER CONF | listener_address | Local listening address (usually primary hostname) | $DBVISIT_BASE/dbvserver/conf | dbvserver.conf |
Standby v7:
File | Parameter Name | Description | File Location | File Name |
DDC FILE | SOURCE | Primary server name with primary database | $DBVISIT_BASE/standby/conf | dbv_<DDC>.env |
DDC FILE | DESTINATION | Standby server name with standby database | $DBVISIT_BASE/standby/conf | dbv_<DDC>.env |
DBVNET CONF | bind_ip | Local listening address (usually primary hostname) | $DBVISIT_BASE/dbvnet/conf | dbvnet.conf |
DBVSERVER CONF HTTPS | bind_ip | Local listening address (usually primary hostname) | $DBVISIT_BASE/dbvserver/conf | dbvserver.conf |
DBVSERVER CONF HTTPS | bind_ip | Local listening address (usually primary hostname) | $DBVISIT_BASE/dbvserver/conf | dbvserver.conf |
Standby v6:
File | Parameter Name | Description | File Location | File Name |
DDC FILE | SOURCE | Primary server name with primary database | $DBVISIT_BASE/standby | dbv_<DDC>.env |
DDC FILE | DESTINATION | Standby server name with standby database | $DBVISIT_BASE/standby | dbv_<DDC>.env |
DBVSERVER CONF HTTPS | bind_ip | Local listening address (usually primary hostname) | $DBVISIT_BASE/dbvserver/etc | dbvserver.conf |
DBVSERVER CONF HTTPS | bind_ip | Local listening address (usually primary hostname) | $DBVISIT_BASE/dbvserver/etc | dbvserver.conf |
If you are not sure, you can always fill in IP address everywhere instead of hostname. It makes the settings more transparent and usually it reveals the potential problems.
There is one thing to remember, if you change the SOURCE/DESTINATION in the DDC file in existing configuration in v8. You have to keep in mind, that everything stored in GUI is pointing to original value (hostname) - including the hosts which are added to your landscape. So after changing the SOURCE/DESTINATION in DDC file, you will have to add the hosts again - this time only via IP address. And afterwards import your DDC file configuration again.
Comments