14

My problem is finding a way to SSH into a Linux server (Ubuntu 18.04) whose IP address changes daily.

I have a client who I occasionally help with admin tasks. I need to ssh into the machine when they need help, but they don't have a static IP, so the public IP of the server is constantly changing. I created a small script to report the IP address of the Linux machine, and find that it changes once per day around noon.

I'm able to setup SSH, and it works both locally and remotely... until the IP address changes. Once that happens, I'm no longer able to connect remotely even using the new IP address.

  • Do I need to restart the SSH service every time the IP address changes?
  • If so, why?
  • Is there any other action I need to take when the IP address changes in order to allow SSH access?

UPDATE

To be clear, my problem is NOT finding the new IP address. I have a script to do that already. The problem is the server stops responding once the IP changes, even though I try to connect using the new IP address.

If I restart the SSH service on the target machine, I have remote access again. But I don't understand why I should have to do this. I would like to know the root cause in hopes of finding a better solution.

Most people seem to think SSH should work as long as we know the new IP, so is this something unique to 18.04? I installed this server for the client recently, so all config settings are still default. (The wouldn't know how to change it.)

12
  • 8
    Use dynamic dns. Nov 12, 2018 at 21:44
  • 12
    Yell at their ISP. Changing IP address once a day is absurd for a business connection, and it's even relatively unheard of for a residential connection. Nov 12, 2018 at 22:00
  • 2
    You say ssh access stops working when the IP address changes. But you haven't said when it starts working again. I assume it must somehow start working again, otherwise you couldn't see it stop working every day. And there isn't anything you are supposed to do when the IP address changes. The ssh server will immediately be reachable on the new IP address.
    – kasperd
    Nov 12, 2018 at 23:07
  • 1
    There could be some weirdness going on like DDNS running, and something like ListenAddress <dynamic host name> in the sshd config file, maybe because someone didn't want internal users being able to ssh into the server. This would actually require restarting the ssh server every time the IP changes. Nov 13, 2018 at 7:03
  • 1
    @MichaelHampton: Yelling will probably not help, paying will. With the current scarcity of IPv4 addresses, most providers charge extra for a static IPv4 address (some even charge extra for a non-static public IP address, otherwise you get Dual Stack Lite). Of course, if OP is on IPv6, things change...
    – sleske
    Nov 14, 2018 at 7:26

8 Answers 8

22

The other answers seem to have overlooked one thing in your question:

Once that happens, I'm no longer able to connect remotely even using the new IP address

DDNS will help you find out the new IP address, but that doesn't seem to be the problem here.

Unfortunately, the server getting a new IP should not be a problem in the standard setup where the ISP provides a router, the server has an internal address behind the router, and the router does the port forwarding. You might need to provide more information about the network topology to get a good answer.

What I could imagine is the server not being behind a router and doing its own PPPoE connection, and a) the ssh server binding to the specific interface address at the point of server restart, b) a firewall on the machine that permits incoming ssh to just the server's IP with the firewall not updating when the IP changes.

To check the first case, do a netstat -nta | grep -w 22 | grep LISTEN. If it says 0.0.0.0:22, that's ok; if it lists a specific IP, then check the sshd config file (/etc/sshd.conf) for ListenAddress.

To check the second case, do iptables -L -n and check if one of the rules in the INCOMING chain matches your server's IP and port 22.

If one of these has the current server address, you'll need to either change it to 0.0.0.0 (make sure you know about the security implications), or update the rule/configuration whenever the IP changes.

Edit

As the server is behind a router, the above ideas probably don't apply(*). In this setup, the router has an external IP (which changes daily), and your internal devices should have some 10.x.y.z or 192.168.x.y addresses which shouldn't change. You connect to the external address, and the router should a port-forwarding rule to the internal address.

This port forwarding shouldn't break when the external IP changes (existing ssh connections will be dropped however), but maybe it's a rule that wasn't setup by you, but by some UPNP magic, with the router dropping UPNP forwards when it gets a new address, and sshd only invoking the rule when it restarts. Did you set up a port forwarder within the router yourself.

Or, it's the internal IP of the server that changes - in that case, something is severely broken with your DHCP. Give your server a fixed internal address.

Or, are you using IPV6? There are some configurations where a device keeps changing its IP to make it less easy to track. See, for example, https://www.internetsociety.org/blog/2014/12/ipv6-privacy-addresses-provide-protection-against-surveillance-and-tracking/ - but in that case, shame on you for not mentioning it in your original post. This might mean your router not doing NAT at all, and my original ideas still being valid even behind a router.

2
  • 3
    +1 for being the first answer to actually address what was asked in the question.
    – kasperd
    Nov 13, 2018 at 15:13
  • @Guntram Blohm Thanks for the answer! The server is behind a router. Would these two possibilities still hold true in that scenario? If so, I'll check them both in the next day or so when I can locally access the target machine.
    – Richard
    Nov 13, 2018 at 15:26
11

Dynamic DNS is one option, another is having the server mail or otherwise send you it’s IP. A simple HTTP call would do (to an endpoint you control and log requests on).

Solving the whole public networking the the other way around is possible as well; you could have the server setup a reverse tunnel, or a VPN connection, which will not be affected by the IP change.

Regarding services not responding on the new address: this completely depends on your network setup. For example: WAN IP on an internal interface via DHCP and an SSH server set to only listen to the IP on your interface known at startup would mean sshd has to be restarted on interface changes.

1
  • 1
    As described in the question there is already a solution in place to find the new IP address. Thus adding dynamic DNS would not really solve any problems. The problem as described is that the ssh server stops responding when the IP address changes. You haven't addressed that problem. VPN connections and reverse tunnels are OTOH expected to stop working when the IP changes, so you need something in place to restart those automatically.
    – kasperd
    Nov 13, 2018 at 15:10
7

You should really look into ddns services. As far as remotely being able to connect to a certain machine with dynamic ip address; ddns is the most commonly used solution.

head over to https://noip.com and signup for an account (it's, errr... supposedly..coff..cof .. Free for 1-3 machines running on the same network(if im not mistaken, dont quote me here: its been a while since ive trusted any of these 'free' services... ). There are also other alternatives such as Afraid DNS (https://freedns.afraid.org/ And even, Cisco's, Open DNS : could also be utilized (provided that this is not you're only client, i'd suggest you sign up for an umbrella account trial, take it for a spin and later sign-up for the real deal >>> they even got one of them GUI-like downloadable extensions which would automatically renew the the ddns hostname whenever your clients ip change. It really is, the easiest, non-tech savvy way that i'm aware of[just in case you gotta call up one of your clients and ask'm to download the GUI rather .... ])

1
  • 3
    While this is certainly helpful for accessing a host on changing IP addresses, that isn't what the question is asking about. The question is why the ssh server stops responding when the IP address changes and how to fix that. A dynamic DNS service does not address that.
    – kasperd
    Nov 12, 2018 at 23:09
4

I'm able to setup SSH, and it works both locally and remotely... until the IP address changes. Once that happens, I'm no longer able to connect remotely even using the new IP address.

Sometimes, it takes a while for DHCP changes to take effect. Try to recycle the DHCP client on the target machine

$> sudo dhclient -r
$> sudo dhclient
Do I need to restart the service every time the IP address changes?
If so, why?

No. You only need to recycle your ssh service when the configuration changes (/etc/ssh/sshd_conf).

Is there any other action I need to take when the IP address changes in order to allow SSH access?

No.

I have a solution that assumes you have sendmail set up on your target machine.

This script sends an email that shows the IP address that the world thinks we have (Thanks ipify.org). Email will always have the most current IP address.

  1. Create dhcp-notify bash script (without the .sh extension)
  2. Put the script in /etc/dhcp/dhclient-exit-hooks.d

    #!/bin/sh
    case "$reason" in (BOUND|RENEW|REBIND|REBOOT)
    (
       echo To: [email protected]
       echo From: [email protected]
       echo "Content-Type: text/html; "
       echo Subject: DHCP reason: $reason
       echo
       echo Your IP address is: `curl -s https://api.ipify.org`
    ) | sendmail -t
    ;;
    esac
    

If this doesn't work, you could always set up a cron to send you the current IP address (lose the case statement).

#!/bin/sh
(
   echo To: [email protected]
   echo From: [email protected]
   echo "Content-Type: text/html; "
   echo Subject: IP address change
   echo
   echo Your IP address is: `curl -s https://api.ipify.org`
) | sendmail -t
5
  • I was just about to suggest this, it's a great idea, and the first thing which came to mind when I read the question. Also, if you're having issues with api.ipify.org (it's blocked on my school network for some reason), you can use this other site which I made a few months previously which does the same thing: findip.win
    – undo
    Nov 13, 2018 at 8:35
  • @karlchilders Thanks for the answer! I already have a cron script reporting the ip address to me, but I had no idea about dhclient hooks. That is awesome! I would much rather only have the IP reported when it actually changes. Question: If this script is able to work and report the IP address, why would recycling dhclient have an effect on SSH access?
    – Richard
    Nov 13, 2018 at 15:37
  • @karlchilders Or does this script only work when you manually recycle dhclient?
    – Richard
    Nov 13, 2018 at 15:54
  • dhclient -r; dhclient will renew the dhcp lease so your host is current. The hooks will work any time a dhcp event occurs either manually or otherwise. Nov 14, 2018 at 20:54
  • @Richard Please mark one of the answers as the preferred answer. Thank you. Nov 19, 2018 at 17:40
3

Thinking outside the box - could you arrange a fixed IPv6 address? It's usually only IPv4 addresses that need to change, due to their scarcity.

6
  • The IP address that I pull down is actually IPv6 every time.
    – Richard
    Nov 13, 2018 at 17:27
  • 1
    @Richard: Sorry to hear it doesn't work for you. I'll leave the answer up for other people who have this problem with IPv4.
    – MSalters
    Nov 13, 2018 at 17:29
  • @Richard You should really mention in your question that you are dealing with IPv6. This makes the problem quite different.
    – Dubu
    Nov 14, 2018 at 8:28
  • 1
    @Dubu It does indeed make the problem different. This kind of problem just isn't supposed to happen on IPv6, which may be why everybody assumed it was IPv4. But in fact you could get some of the described symptoms if you mistakenly use a privacy address rather than a static address. If for example you ask a website what your IP address is, it will show your privacy address rather than your static address. And using that for ssh connections is not a good idea. Still doesn't explain why it doesn't work until the ssh server has been restarted.
    – kasperd
    Nov 14, 2018 at 10:24
  • 1
    @Dubu: That's a rather non-standard approach. DHCPv6-PD (Prefix Delegation) in combination with DHCP Unique ID's (DUID) should prevent that.
    – MSalters
    Nov 14, 2018 at 15:32
1

What I've been doing for almost a year. I ran into your problem in January this year while giving a speech at my local university.

This script has been running on my machine ever since then: //Pretty much self-explanatory//

import smtplib
from requests import get
import time

user = '[email protected]'
psd = 'examplepassword'
msg = 'Hey, your ip has changed! Use this one from now on: '
currip = '0.0.0.0' # It'll send an email the first time you execute this aswell
while True:
    newip = get('https://api.ipify.org').text
    if currip == newip:
        print("nonewip")
        #You can just comment the line above this one if you want to
    else:
        tmpmsg = "\n"+msg + newip +" The old one used to be: "+currip
        currip = newip
        server = smtplib.SMTP('smtp.gmail.com', 587)
        server.connect('smtp.gmail.com')
        server.ehlo()
        server.starttls()
        server.ehlo()
        server.login(user,psd)
        #You can change the second parameter, use ('from','to','message')
        server.sendmail(user, user, tmpmsg)

        server.quit()
        with open('/home/pi/DEV/iphistory.txt', 'a') as f:
            f.write('newip: '+currip+'\n')
            f.close()
        print('New IP Found: '+tmpmsg)
    time.sleep(7200)

Python 3.x should work flawlessly Maybe not the best solution out there, but it works.

You'll get an email in your Inbox each time the machine's public IP changes.

Now, regarding your questions:

Do I need to restart the service every time the IP address changes? If by restarting the service you mean re-establishing the ssh connection, yes.

If so, why? Because if the address you're trying to communicate with no longer provides the service you're looking for. It's not your machine anymore.

Is there any other action I need to take when the IP address changes in order to allow SSH access? Just SSH to the new address.

Cheers! JSR

4
  • Thanks for the answer @JSR! I have actually written a similar script to report the IP address to me. But once it changes, even knowing the new IP address, I'm unable to SSH into the remote machine. Have you encountered that issue?
    – Richard
    Nov 13, 2018 at 15:43
  • Are you sure you're retrieving the right address? if so, try to ping at it once you get the IP. I've never seen that issue before Also, just in case you didn't know, you can run the script on background with "python3 scriptname.py &" Also, check the forwarding ports of the router, I assume the local IP doesn't change, but still it's worth checking out. Nov 13, 2018 at 15:47
  • Yes, the IP is correct. If I restart the SSH service on the target machine, I'm able to connect again. But I don't understand why that should be the case.
    – Richard
    Nov 13, 2018 at 16:01
  • @Richard the issue here is that since your IP changes, your host stops every connection since the RSA Key you're using to authenticate does not match with the one it's requesting. I'm by no means an expert on this topic, so don't take my words as a fact... But your situation is like that PROBABLY because of what I just said. Nov 14, 2018 at 6:50
1

It is worth viewing this from a different side: usually, it's much simpler to create an outward connection from the machine that requires your attention (DNS, NAT and other firewall settings do not matter at all or are much simpler).

You could use this to establish a bullet-proof and simple solution to get into a remote machine R. The only requirement is that you can provide public ssh access to one of your own local machines (let's call it S). Then proceed as follows:

  1. Create outward ssh connection from R to S, establishing a reverse tunnel back into R:

    ssh -L 22:<address-of-S>:22000

  2. On S, use the reverse tunnel to ssh into the remote machine R:

    ssh -p 22000 127.0.0.1

Step 1 could be triggered manually and on-demand by the remote side whenever your assistance is needed. Alternatively, you could create a service on R that will continously maintain such a reverse tunnel to S.

I used such a setup to log into remote (mobile) systems that were behind firewalls/NAT and which had no DNS entries at all.

1

If you just need remote login, use mosh. Apart from being able to cope excellently with IP changes (with almost no latency during the switch) it has also other advantages over plain ssh, like local predictive echo, low latency, way faster recovery from broken links.

If you specifically need ssh (say, you need X11 forwarding or something), I suggest setting up VPN (e.g. OpenVPN), preferably over UDP, with a short keepalive. TCP connections (i.e. your ssh) over VPN will heal and remain connected after IP change, it will just take longer (up to a minute or so), but you can play with /proc/sys/net/ipv4/tcp_* entries to make it more acceptable.

Edit:

  • mosh needs ssh for authentication, but once authenticated, the connection stays until you log out (or reboot), and you can investigate the failure of ssh at your leisure (e.g. strace -f -p pid_of_sshd)
  • it is possible to use mosh without ssh, answer adapted from here:

On the server run:

mosh-server new -p $randomport -- $shellprogram

you get an result like QzdRHbAWzL7eRobi75DCrz

On the client run:

MOSH_KEY=QzdRHbAWzL7eRobi75DCrz mosh-client $serverip $radomport

Note that $serverip has to be an ip, no hostname resolution.

How you get the key from one side to the other is up to you. I'd suggest encrypting with a pre-shared key and instant-messaging, mailing, or making a local user dictate it via phone call.

  • if there is really a problem with ssh, install inetd and run ssh from there, not as a standalone daemon, e.g. by this configuration line for "classic" inetd.conf, this will run a new daemon each time you log in (note that not all inetd forks handle ipv6):

ssh stream tcp6 nowait root /usr/sbin/sshd sshd -i

2
  • 1
    Thanks for the answer @Radovan! Mosh looks interesting, but it looks like it uses SSH to authenticate. So the root of my problem would still exist, correct? Let me know if I have misunderstood the docs.
    – Richard
    Nov 15, 2018 at 18:15
  • @Richard Yes, however there are possibilities... I've edited my answer. Nov 16, 2018 at 9:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .