[sdnog] tips and best practices

[Nishal Goburdhan]

On 8 Jul 2015, at 21:30, Hiba Eltigani wrote:
> On 8 July 2015 at 12:51, Samir S. Omer <samir.saif at sudren.edu.sd> 
> wrote:
>> Dear SdNOGers
>> 1- I usually edit my local 'host' file with the hostnames with their
>> respected IPs of the machine I administer so that i don't have to 
>> remembers
>> the IPs.

that’s both good and bad.  my general belief is that i should be able 
to work from (almost) anywhere/anything.  so i’d strongly suggest 
coming up with “tips” that will help you regardless of the 
environment that you’re working from.

personally, i would suggest that if you don’t want to remember the IP 
addresses,
* then make sure that everything is put into DNS correctly.
* make sure the search-domain of your management host, has the domain 
you need in the search-list.
* make sure you use a set of trusted hosts *only* for management 
purposes.  and make sure that your path to these hosts are trusted.  so, 
for example, from your laptop/desktop, you SHOULD (2119-style) be able 
to use primarily key-based authentication to connect to a secure 
management host.  (personally, i think that SHOULD, should be a MUST).  
that connection, if you are coming from an external network MUST be over 
some sort of secure channel (IPSEC/TLS/..blah).
* and, of course, you MUST limit which networks/hosts can connect to 
your routers/machines/whatever for management purposes.  there’s no 
point forcing everyone to ssh to “admin.sdnog.sd” to do work, when 
all your routers/servers are open to the world, management-wise :-)

this scales not only for you;  it scales for the rest of your team too, 
and makes it easier to manage security policies.


>> 2- when I'm configuring any monitoring or alerting systems with
>> auto-generated emails, I usually put an appropriate identifier within
>> square brackets
>> to help me filter and auto-classify email to their custom folder in 
>> my
>> mailbox.

yes, good tip.  also depending on how critical this is to you, consider 
using an internal, and external domain, and host for sending/receiving 
monitoring and management.  if your single domain has a problem, 
you’ll be certain that you know about it.  it increases your overall 
complexity a little (you now have two domains to run, administer, and 
pay for) but it’s helped me several times.


>> 3- my favorite terminal is MobaXterm, it has many useful features 
>> that
>> makes it easier for managing machines 
>> "http://mobaxterm.mobatek.net/".

whoa!  that’s a lot more than just a terminal emulator.  my rule of 
thumb when using software is to try to get the specific task that i want 
to get done, done well.  (a spin off from the UNIX philosophy [1]).  
running a full on X-server might not be what you want, if you just want 
a terminal session.

an important thing to remember is that you should also be comfortable on 
a “vanilla” system.  building your work environment to be efficient 
around *your* workstation is one thing.  but you SHOULD be able to work, 
from another.  the classic example i use is “vi” on a *nix host, vs. 
something like ‘ee’  (which is much easier to use).  almost every 
*nix distribution will ship with “vi” or “vim” installed, so, if 
you arrive onsite somewhere, and don’t have anything else to work on, 
you should know how to use vi, to be productive.
for that reason, on my OSX laptop, i use pre-installed utilities that i 
know that any other OSX host will have:
* terminal;  for remote ssh/telnet (even though my friends tell me that 
iTerm is nicer for them)
* cu;  for terminal emulation, or what you would call console access 
(instead of, say, some 3rd party app)
.. etc.
… and for when i need to use a windows system, i generally have a copy 
of Putty on a clean memory stick that i leave behind.

<tl;dr>  tools are great for power-users, but it’s always best to know 
how to use the basics of a system, since those tools will likely always 
be available.



>> 4- as most of us use Windows based machines in Sudan, I usually 
>> install
>> dig in my machine so that I can quickly troubleshoot any DNS issues. 
>> "
>> http://nil.uniza.sk/windows/windows-7/how-install-dig-dns-tool-windows-7"
>>
>> 5- I use KeePass to manage and store all my passwords. "
>> http://keepass.info/"

this is really good advice.  everyone should get out of the habit of 
re-using passwords.  even if you think you have a super-secure 
36-character password with mixed case, numbers, and special characters, 
all it takes it is a break into one system that didn’t store this 
properly [2] to have this out in the open.  and with the proliferation 
of data on today’s social networks, figuring out answers to common 
“security questions” is trivial for most determined folk.

so, *unique*, cryptographically strong passwords for every 
service/sign-in that you use, is a MUST, if you can’t get access to 
2factor authentication systems easily.


>> these are some of the methods and tools I use, I hope it helps you, 
>> and
>> looking forwards to hearing about your tips and best practices

perhaps, we can get some volunteers to collate, and add to the sdnog 
wiki at wiki.sdnog.sd ;-)


> I can add few more:
> 1- Whenever possible, I try to make my services run under separate 
> user
> that's not root.
>
> 2- MRTG and Syslog are my essential monitoring tools specially if I
> manage to get memory and    CPU utilization monitored. In addition to 
> that
> I used ActiveXperts (windows) for links availability it send you SMS 
> when
> ping to one of your nodes fail. I think Nagios can do the same for 
> linux
> platforms but I have never used it before.
>
> 3- Cron along with shell scripts are simple way to automate back-ups, 
> I
> just need to make sure proper security is in place.
>
> 4- I also have the habit of preparing all the new configuration in 
> text
> files along with the roll-back configuration before applying any 
> changes to
> the network node. It helps minimize the errors and down time.
>
> 5- terminal servers are must to have specially for remote location, I 
> just
> need to make sure that they are connected through separate link for 
> obvious
> reasons :).


the best tip that i can give you (for anything that you’re doing) is 
what some of you may already have heard me say in class previously : the 
key to success is planning.  it’s great to be agile, and good 
engineers are able to react, and create workarounds on the fly, but 
that’s not a reason to have a well laid out plan that clearly says:
* what you are aiming to accomplish  (ie. the problem statement)
* what’s the expected impact
* what’s the workaround (if any)
* what’s the fall-back plan.


—n.


[1]  https://en.wikipedia.org/wiki/Unix_philosophy
[2]  https://en.wikipedia.org/wiki/2012_LinkedIn_hack