System Monitoring with Xymon/Administration Guide: Difference between revisions

Browse history interactively

[unreviewed revision]

← Older edit Newer edit →

Content deleted Content added

VisualWikitext

Inline

Revision as of 21:03, 12 February 2008

All things related system administration will be documented here.

Architecture of a Hobbit System Monitoring Environment

TBC

Picking an OS for Hobbit Server

These some notes and advices from Hobbit users.

Linux

- Debian
- RedHat Linux

Solaris 11/06 U3 on Sun hardware

Pros

- Plus 1: Turbocharged TCP/IP.
- Plus 2: dtrace
- Plus 3: Self Heal

Cons

Minus 1: Hobbit depended other open source software doesn't come with Solaris by default. Following are three sources that you can get the software in binary or source code format.

http://www.blastwave.org
http://www.sunfreeware.com has lots of open source.
http://www.thewrittenword.com

List of software required to meet all dependecies and order of installation:

common-1.4.5-SunOS5.8-sparc-CSW.pkg.gz
pcre-4.5-SunOS5.8-sparc-CSW.pkg.gz
fping-2.4,REV=2004.10.12_rev=b2_to_ipv6-SunOS5.8-sparc-CSW.pkg.gz
zlib-1.2.3,REV=2007.05.12-SunOS5.8-sparc-CSW.pkg.gz
png-1.2.18-SunOS5.8-sparc-CSW.pkg.gz
libiconv-1.9.2-SunOS5.8-sparc-CSW.pkg.gz
expat-1.95.7-SunOS5.8-sparc-CSW.pkg.gz
ggettext-0.14.1,REV=2005.06.29-SunOS5.8-sparc-CSW.pkg.gz
libpopt-1.7,REV=2004.05.15-SunOS5.8-sparc-CSW.pkg.gz
chkconfig-1.2.24h,REV=2006.12.12-SunOS5.8-sparc-CSW.pkg.gz
libpopt-1.7,REV=2004.05.15-SunOS5.8-sparc-CSW.pkg.gz
openssl-0.9.8,REV=2007.05.10_rev=e-SunOS5.8-sparc-CSW.pkg.gz
imaprt-2004,REV=2006.09.02_rev=g-SunOS5.8-sparc-CSW.pkg.gz
freetype2-2.1.10,REV=2005.12.11-SunOS5.8-sparc-CSW.pkg.gz
libart-2.3.16-SunOS5.8-sparc-CSW.pkg.gz
berkeleydb44-4.4.20,REV=2007.01.27-SunOS5.8-sparc-CSW.pkg.gz
ncurses-5.5,REV=2006.02.10-SunOS5.8-sparc-CSW.pkg.gz
readline-5.0,REV=2005.06.07-SunOS5.8-sparc-CSW.pkg.gz
gbc-1.06-SunOS5.8-sparc-CSW.pkg.gz
gdbm-1.8.3,REV=2006.01.01-SunOS5.8-sparc-CSW.pkg.gz
perl-5.8.8,REV=2007.03.16-SunOS5.8-sparc-CSW.pkg.gz
cvs-1.11.22-sol10-sparc-local.gz
rrdtool-1.2.19,REV=2007.02.07-SunOS5.8-sparc-CSW.pkg.gz
libnet-1.0.2,REV=2004.04.08_rev=a-SunOS5.8-sparc-CSW.pkg.gz
berkeleydb4-4.2.52,REV=2005.04.28_rev=p4-SunOS5.8-sparc-CSW.pkg.gz
sasl-2.1.22,REV=2007.06.19-SunOS5.8-sparc-CSW.pkg.gz
openldap_rt-2.3.35,REV=2007.04.14-SunOS5.8-sparc-CSW.pkg.gz
hobbit-4.2.0,REV=2007.04.12-SunOS5.8-sparc-CSW.pkg.gz
hobbit_client-4.2.0,REV=2007.04.12-SunOS5.8-sparc-CSW.pkg.gz

Notes

To avoid "hobbitd status-board not available" error message in bbgen webpage, add "set ip:do_tcp_fusion = 0x0" into /etc/system to disable TCP fusion.
1. References: http://www.hswn.dk/hobbiton/2007/04/msg00187.html
2. Solaris 5.10 kernel patch 120011-14-1, it fix this bug "6449337 kmem exhaustion caused by tcp fusion flow control logic error" .

Hobbit Server: Solaris Intel 11/06 U3 VMware appliance on a 2GB flash pen drive

Following are main procedures for this to-go hobbit server.

VMware server 1.0.1 to create Solaris 10 VMware session.
Create a 1.9G partition, select custom install.
modify the partition table to take out /export/home,only leave /swap and /.
decrease default 512M swap size to 300M.
select "Core group" (about 573M in size).
Install httpd server
Install hobbit server

Hobbit Server and Development: Solaris Intel 11/06 U3 VMware appliance on a 4GB flash pen drive

VMware server 1.0.1 to create Solaris 10 VMware session.
- Need to use vmware player 1.0.3 so dhcp will work.

Hobbit Server Test site

Solaris Intel 11/06 U3 VMware appliance on a 4GB flash pen drive

Operational difference between Hobbit and BB BTF

Servers

This is a comparison table on how Hobbit server is different from BB when performing an adminstration task.

Operation	Hobbit 4.2.0 above	Big Brother BTF(Better Than Free, 1.9c version above)
start/stop server	~/hobbit.sh start/stop	~/runbb.sh start/stop
Delete a host	~/bin/bb 127.0.0.1 "drop HOSTNAME [test]"	$BBHOME/bin/bbrm
add a host	1. add hostnames into bb-hosts	1. add hostnames into bb-hosts
Log data path	1.	1.

Clients

This is a detail comparison on how Hobbit is different from BB when performing an adminstration task.

Operation	Hobbit 4.2.0 above	Big Brother BTF(Better Than Free, 1.9c version above)
addin external module	~hobbit/client/etc/hobbitclient.cfg	$BBHOME/etc/bb-extab

References

http://www.hswn.dk/hobbit/help/about.html

Capacity Planning

rule of Thumb is 5MB per server being monitored

Installation

Install premade hobbit server package for your OS

AIX ( http://rpm.razorsedge.org/ )
Debian ( http://packages.debian.org/etch-backports/hobbit )
FreeBSD
HP-UX ( http://www.razorsedge.org/~mike/software/hobbit/hpux/ )
IRIX
RedHat Linux / RedHat Enterprise Linux / Fedora Core ( http://rpm.razorsedge.org/ )
Solaris ( http://www.blastwave.org/ )

Install premade hobbit client packages for your OS

AIX ( http://rpm.razorsedge.org/ )
Debian ( http://packages.debian.org/etch-backports/hobbit-client / http://packages.debian.org/lenny/hobbit-plugins )
FreeBSD
HP-UX ( http://www.razorsedge.org/~mike/software/hobbit/hpux/ )
IRIX
RedHat Linux / RedHat Enterprise Linux / Fedora Core ( http://rpm.razorsedge.org/ )
Solaris ( http://www.blastwave.org/ )
NSLU2 Unslung OS.

bash-2.05b# ls -lrt
-r-xr-xr-x    1 root     administ     2891 Aug  9  2006 hobbitclient.sh
-r-xr-xr-x    1 root     administ     3033 Aug  9  2006 hobbitclient-sunos.sh
-r-xr-xr-x    1 root     administ     1841 Aug  9  2006 hobbitclient-sco_sv.sh
-r-xr-xr-x    1 root     administ     1701 Aug  9  2006 hobbitclient-osf1.sh
-r-xr-xr-x    1 root     administ     1904 Aug  9  2006 hobbitclient-openbsd.sh
-r-xr-xr-x    1 root     administ     1907 Aug  9  2006 hobbitclient-netbsd.sh
-r-xr-xr-x    1 root     administ     2512 Aug  9  2006 hobbitclient-linux.sh
-r-xr-xr-x    1 root     administ     1834 Aug  9  2006 hobbitclient-irix.sh
-r-xr-xr-x    1 root     administ     2070 Aug  9  2006 hobbitclient-hp-ux.sh
-r-xr-xr-x    1 root     administ     2039 Aug  9  2006 hobbitclient-freebsd.sh
-r-xr-xr-x    1 root     administ     1554 Aug  9  2006 hobbitclient-darwin.sh
-r-xr-xr-x    1 root     administ     1971 Aug  9  2006 hobbitclient-aix.sh
-rwxr-xr-x    1 root     root       832531 Feb 16 16:51 bb
-rwxr-xr-x    1 root     root       695294 Feb 16 16:51 hobbitlaunch
-rwxr-xr-x    1 root     root       676992 Feb 16 16:52 bbcmd
-rwxr-xr-x    1 root     root       842123 Feb 16 16:52 bbhostgrep
-rwxr-xr-x    1 root     root       670898 Feb 16 16:52 bbhostshow
-rwxr-xr-x    1 root     root       716800 Feb 16 16:52 bbdigest
-rwxr-xr-x    1 root     root       944795 Feb 16 16:53 logfetch
-rwxr-xr-x    1 root     root       839071 Feb 16 16:53 clientupdate
-rwxr-xr-x    1 root     root       830390 Feb 16 16:53 orcahobbit
-rwxr-xr-x    1 root     root       698541 Feb 16 16:53 msgcache
bash-2.05b# ./bb
Hobbit version 4.2.0
Usage: ./bb [--debug] [--proxy=http://ip.of.the.proxy:port/] RECIPIENT DATA
  RECIPIENT: IP-address, hostname or URL
  DATA: Message to send, or "-" to read from stdin
bash-2.05b# uname -a
Linux LKG7BFA96 2.4.22-xfs #1 Sun Jun 12 21:17:17 PDT 2005 armv5b unknown
bash-2.05b# date
Sat Feb 17 11:45:50 CST 2007
bash-2.05b#

NSLU2 Debian OS:

Compiling from tar ball

Building from package source using TWW HPMS

TWW Hyper Package Management system can help a software developer or system administrator to create different native package formats for different OS. The package source for compiling and packaging hobbit client and server software are in XML format that can be repeated reliably with TWW's sb and pb tools.

Hobbit server and Hobbit client package source is GPL licensed on TWW's support ftp server.

Building from src RPM

Sometimes it's better to build your own RPMs specifically for your environment. If you are using RH Enterprise or CentOS, the Fedora Core or generic RPM may not install correctly. You could also run into this problem if you have versions of dependant libraries that are not compatible with the system that the RPM was built on.

In order to build the src RPM, you'll need several packages:

openssl-devel, openldap-devel, and pcre-devel from the CentOS CDs.

  You may also have to make a link from /usr/include/pcre/pcre.h to /usr/include/pcre.h

rrdtool-devel

  I recommend getting this from the DAG repository ( http://dag.wieers.com/packages/rrdtool/ )

fping

  Also available from the DAG repository ( http://dag.wieers.com/packages/fping/ )

RPM's from a matching version of RH EL, usually work on Centos with no problem (for example RPMs for EL 4 work fine on Centos 4)

Once you have all the depenencies installed, download the src RPM from SourceForge ( http://sourceforge.net/projects/hobbitmon/ ). Once you have that, just run rpmbuild --rebuild hobbit-xxxx.src.rpm. For example:

   rpmbuild --rebuild hobbit-4.1.0-1.src.rpm

The rpmbuild command should compile and build the RPM for you. You can watch the compiler output for any problems. After it is done, you should have new RPMs in the /usr/src/redhat/RPMS/i386 directory (assuming your architecture is i386). This process will build both server and client RPMs for your system. The server RPM also includes the client, so it is not necessary to install both of them.

Hobbit Configuration and tuning

Hobbit(bb) port 1984 encryption

References: http://www.stunnel.org/

Plain text bb message will be a bottleneck to make Hobbit a enterprise solution which require high security standard. Following is an attempt to make your CIO smile on hobbit solution.

Machine A : has both HB Server and Stunnel server running.
Machine B : is a BB client.
Machine C : is a hobbit client with stunnel client enabled. hb client will send bb message via encrypted port 1999.
Machine D : is a HB client.
Note: old bb port is one way, hb's bb protocol's is bi-directional.

      Machine A (192.168.1.111)                                          

    ---------------------------
     HB Server process         |   <---------port 1984 <---------  BB client (Machine B)
         |                     |
         |1984                 |   <---------port 1984 --------->  HB client (Machine D)
         |                     |                                   
   Stunnel Server process 1999 |   <-------- port 1999 ----------> 1999 Stunnel Client
   ----------------------------                                    |            (Machine C 192.168.1.141)
                                                                   |
                                                                   --1984 ---HB client

Configure stunnel server to run in hobbit server

stunnel config file on server to direct 1999 into local 1984 port.

accept = 1999, we accept any incoming bb message on port 1999.
connect = 127.0.0.1:1984, redirect 1999 to 1984 on hb server itself.

bash-3.00# cat /opt/stunnel420/etc/stunnel/stunnel.conf
<snip>
[hobbit-server]
accept  = 1999
connect = 1984
<snip>
bash-3.00#

starting stunnel server on machine A. we can see hobbit-server port redirection is ok.

bash-3.00# /etc/init.d/stunnel420 start
Starting universal SSL tunnel: stunnel2007.04.29 06:47:50 LOG7[1898:1]: RAND_status claims sufficient entropy for the PRNG
2007.04.29 06:47:50 LOG7[1898:1]: PRNG seeded successfully
2007.04.29 06:47:50 LOG7[1898:1]: Certificate: /opt/stunnel420/etc/stunnel/stunnel.pem
2007.04.29 06:47:50 LOG7[1898:1]: Certificate loaded
2007.04.29 06:47:50 LOG7[1898:1]: Key file: /opt/moto/stunnel420/etc/stunnel/stunnel.pem
2007.04.29 06:47:50 LOG7[1898:1]: Private key loaded
2007.04.29 06:47:50 LOG7[1898:1]: SSL context initialized for service pop3s
2007.04.29 06:47:50 LOG7[1898:1]: Certificate: /opt/stunnel420/etc/stunnel/stunnel.pem
2007.04.29 06:47:50 LOG7[1898:1]: Certificate loaded
2007.04.29 06:47:50 LOG7[1898:1]: Key file: /opt/stunnel420/etc/stunnel/stunnel.pem
2007.04.29 06:47:50 LOG7[1898:1]: Private key loaded
2007.04.29 06:47:50 LOG7[1898:1]: SSL context initialized for service hobbit-server
.
bash-3.00#

make sure stunnel is running.

bash-3.00# ps -eaf |grep stunnel
  nobody  1984     1   0 06:55:00 ?           0:00 /opt/stunnel420/sbin/stunnel
    root  2133  1811   0 07:04:32 pts/2       0:00 grep stunnel
bash-3.00#

Testing port 1999 on hb server directly, typing garbage message "asdf" and then control+d to quit.

bash-3.00# telnet machineA.test.com 1999
Trying 192.168.1.111...
Connected to machineA.test.com.
Escape character is '^]'.
asdf
Connection to machineA.test.com closed by foreign host.
bash-3.00#

We can see port 1999 has incoming message from 192.168.1.141(machine c)in stunnel log file on machine A.

bash-3.00# tail -10f /opt/stunnel420/etc/stunnel/stunnel.log
2007.04.29 06:55:00 LOG5[1983:1]: 125 clients allowed
2007.04.29 06:55:00 LOG7[1983:1]: FD 4 in non-blocking mode
2007.04.29 06:55:00 LOG7[1983:1]: FD 5 in non-blocking mode
2007.04.29 06:55:00 LOG7[1983:1]: FD 6 in non-blocking mode
2007.04.29 06:55:00 LOG7[1983:1]: SO_REUSEADDR option set on accept socket
2007.04.29 06:55:00 LOG7[1983:1]: pop3s bound to 0.0.0.0:995
2007.04.29 06:55:00 LOG7[1983:1]: FD 7 in non-blocking mode
2007.04.29 06:55:00 LOG7[1983:1]: SO_REUSEADDR option set on accept socket
2007.04.29 06:55:00 LOG7[1983:1]: hobbit-server bound to 0.0.0.0:1999
2007.04.29 06:55:00 LOG7[1984:1]: Created pid file /stunnel.pid
2007.04.29 06:55:35 LOG7[1984:1]: hobbit-server accepted FD=0 from 192.168.1.141:38764
2007.04.29 06:55:35 LOG7[1984:2]: hobbit-server started
2007.04.29 06:55:35 LOG7[1984:2]: FD 0 in non-blocking mode
2007.04.29 06:55:35 LOG7[1984:2]: TCP_NODELAY option set on local socket
2007.04.29 06:55:35 LOG5[1984:2]: hobbit-server accepted connection from 192.168.1.141:38764
2007.04.29 06:55:35 LOG7[1984:2]: SSL state (accept): before/accept initialization
2007.04.29 06:55:39 LOG3[1984:2]: SSL_accept: 1408F10B: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
2007.04.29 06:55:39 LOG5[1984:2]: Connection reset: 0 bytes sent to SSL, 0 bytes sent to socket
2007.04.29 06:55:39 LOG7[1984:2]: hobbit-server finished (0 left)

Configuring hb client to use port 1999

add hobbitclientLocalIP into hobbitclient.cfg file. We want hobbit client send bb message to itself.


bash-3.00# grep ^BBDISPLAYS   /etc/opt/hobbitclient42/hobbitclient.cfg
BBDISPLAYS="myotherhobbitserver.my.com hobbitclientLocalIP"                   # IP of multiple Hobbit servers. BBDISP must be "0.0.0.0".
bash-3.00#

bash-3.00# egrep -v '^;|^$'  /opt/stunnel420/etc/stunnel/stunnel.conf
cert = /opt/stunnel420/etc/stunnel/stunnel.pem
sslVersion = SSLv3
chroot = /opt/stunnel420/var/lib/stunnel/
setuid = nobody
setgid = nogroup
pid = /stunnel.pid
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
debug = 7
output = stunnel.log
client = yes
[hobbitclient]
connect  =  hbServerRemoteIP:1999
accept   =  hbLocalIP:1984
bash-3.00#

A successful hobbit client stunneling to hobbit server using port 1999.

bash-3.00# grep 06:50   stunnel.log
2007.08.19 00:06:50 LOG7[14842:1]: hobbitclient accepted FD=0 from HobbitclientIP:63758
2007.08.19 00:06:50 LOG7[14842:3]: hobbitclient started
2007.08.19 00:06:50 LOG7[14842:3]: FD 0 in non-blocking mode
2007.08.19 00:06:50 LOG7[14842:3]: TCP_NODELAY option set on local socket
2007.08.19 00:06:50 LOG5[14842:3]: hobbitclient accepted connection from HobbitclientIP:63758
2007.08.19 00:06:50 LOG7[14842:3]: FD 1 in non-blocking mode
2007.08.19 00:06:50 LOG7[14842:3]: hobbitclient connecting HobbitServerIP:1999
2007.08.19 00:06:50 LOG7[14842:3]: connect_wait: waiting 10 seconds
2007.08.19 00:06:50 LOG7[14842:3]: connect_wait: connected
2007.08.19 00:06:50 LOG5[14842:3]: hobbitclient connected remote server from HobbitclientIP:63759
2007.08.19 00:06:50 LOG7[14842:3]: Remote FD=1 initialized
2007.08.19 00:06:50 LOG7[14842:3]: TCP_NODELAY option set on remote socket
2007.08.19 00:06:50 LOG7[14842:3]: SSL state (connect): before/connect initialization
2007.08.19 00:06:50 LOG7[14842:3]: SSL state (connect): SSLv3 write client hello A
2007.08.19 00:06:50 LOG7[14842:3]: SSL state (connect): SSLv3 read server hello A
2007.08.19 00:06:50 LOG7[14842:3]: SSL state (connect): SSLv3 read finished A
2007.08.19 00:06:50 LOG7[14842:3]: SSL state (connect): SSLv3 write change cipher spec A
2007.08.19 00:06:50 LOG7[14842:3]: SSL state (connect): SSLv3 write finished A
2007.08.19 00:06:50 LOG7[14842:3]: SSL state (connect): SSLv3 flush data
2007.08.19 00:06:50 LOG7[14842:3]:    1 items in the session cache
2007.08.19 00:06:50 LOG7[14842:3]:    2 client connects (SSL_connect())
2007.08.19 00:06:50 LOG7[14842:3]:    2 client connects that finished
2007.08.19 00:06:50 LOG7[14842:3]:    0 client renegotiations requested
2007.08.19 00:06:50 LOG7[14842:3]:    0 server connects (SSL_accept())
2007.08.19 00:06:50 LOG7[14842:3]:    0 server connects that finished
2007.08.19 00:06:50 LOG7[14842:3]:    0 server renegotiations requested
2007.08.19 00:06:50 LOG7[14842:3]:    1 session cache hits
2007.08.19 00:06:50 LOG7[14842:3]:    0 session cache misses
2007.08.19 00:06:50 LOG7[14842:3]:    0 session cache timeouts
2007.08.19 00:06:50 LOG6[14842:3]: SSL connected: previous session reused
2007.08.19 00:06:50 LOG7[14842:3]: Socket closed on read
2007.08.19 00:06:50 LOG7[14842:3]: SSL write shutdown
2007.08.19 00:06:50 LOG7[14842:3]: SSL alert (write): warning: close notify
2007.08.19 00:06:50 LOG6[14842:3]: SSL socket closed on SSL_shutdown
2007.08.19 00:06:50 LOG7[14842:3]: Socket write shutdown
2007.08.19 00:06:50 LOG5[14842:3]: Connection closed: 30068 bytes sent to SSL, 0 bytes sent to socket
2007.08.19 00:06:50 LOG7[14842:3]: hobbitclient finished (0 left)
bash-3.00#

32 bit vs 64 bit binary for hobbit on Solaris

This article describe this subject in great detail.

Configuration

LDAP Authentication

Example httpd.conf (Apache 2.0.x with LDAP authenticated against Active Directory):

Substitute LDAPSERVER.DOMAIN.COM with your LDAP server

<USERNAME>: use account with permission to view LDAP directory

<PASSWORD>: password for account (You should limit what this account can do)

<Directory "/var/hobbit/cgi-secure">
    AllowOverride None
    Options ExecCGI Includes
    Order allow,deny
    Allow from all

    AuthType Basic
    AuthName "Hobbit Administration"
    AuthLDAPEnabled on
    AuthLDAPURL ldap://LDAPSERVER.DOMAIN.COM:389/dc=DOMAIN,dc=COM?sAMAccountName?sub?(objectClass=person)
    AuthLDAPBindDN "cn=<USERNAME>,cn=Users,dc=DOMAIN,dc=COM"
    AuthLDAPBindPassword <PASSWORD>
    require valid-user

</Directory>

Same for a Novell-edir ldap server:

<Directory "/usr/lib/hobbit/cgi-secure">
    AllowOverride None
    Options ExecCGI Includes
    Order allow,deny
    Allow from all

    AuthName "Hobbit-Admin"
    AuthType Basic
    AuthLDAPURL ldap://LDAPSERVER.DOMAIN.COM/o=TREE,ou=Users?cn?sub?(groupMembership=cn=your_group,ou=groups,o=TREE)
    require valid-user

</Directory>

Alerts setting

Pager

Using sms_client [smsclient.org]

Create a shell-script (/usr/bin/hobbitsms) like this:


#!/bin/bash
if [ $RECOVERED != 1 ]; then
echo $RCPT \"HOBBIT : $BBHOSTSVC  is $BBCOLORLEVEL\" >> /var/log/hobbit/page.log
/usr/bin/sms_client $RCPT "HOBBIT : $BBHOSTSVC  is $BBCOLORLEVEL"
else
echo $RCPT \"HOBBIT : $BBHOSTSVC  is weer OK\" >> /var/log/hobbit/page.log
/usr/bin/sms_client $RCPT "HOBBIT : $BBHOSTSVC  is OK"
fi

Edit hobbit-alerts.cfg and add the lines for the alerts you want to receive:

      SCRIPT /usr/bin/hobbitsms hobbit DURATION>5 FORMAT=SMS REPEAT=180 COLOR=red TIME=W:0730:1800 RECOVERED

Pager

Using snpp [sendpage.org]

Create a shell-script (/usr/bin/hobbitsnpp) like this:


#!/bin/bash
/usr/bin/snpp -n $RCPT <<SCRIPTEOF
$BBALPHAMSG
SCRIPTEOF

Email

Tuning

How to shorten the ping test time ?

High Availablity with LinuxHA and DRBD

High Availablity on Solaris

SunCluster

Free and opensourced clustering software from Sun. Commercial technical support is available.

Using two sol-nv-b68-x86 VMware sessions with Sun Cluster express 07/07.

References

FST HA

An opensource Clustering solution specifically for Solaris.

FSTha vs other HA

Hobbit and Remedy Ticket System

Overview

Remedy ticket system has a web interface for opening up a ticket to a perticular ticket queue.

Perl approach is to use following software to automate the ticket request upon a alert occured.

perl
LWP
trouble_ticket.tgz on http://www.deadcat.net
an entrance URL on remedy server web interface.
A perl subroutine to open up remedy ticket.

Open Remedy ticket on hobbit alerts

Open Remedy ticket on demand

Migration from BB

Cost (efforts) of Migration

System and Inventory Monitoring

System monitoring and inventory monitoring can achieved by an external module to report a system's inventory's informaton.(TBC)

Trouble Shooting Guide

Q. When I click on a status icon I get the message "Status not available". What should I check?

A. First make sure that the server is actually running.

ps -ef | grep hobbitd

You should see several processes similar to:

hobbit   32717 32716  0 Nov07 ?        00:01:07 hobbitd --pidfile....
hobbit   32726 32716  0 Nov07 ?        00:00:03 hobbitd_channel --channel=page...
hobbit   32727 32716  0 Nov07 ?        00:01:58 hobbitd_channel --channel=status...
hobbit   32728 32716  0 Nov07 ?        00:00:01 hobbitd_channel --channel=data...
hobbit   32725 32716  0 Nov07 ?        00:00:00 hobbitd_channel --channel=stachg...

If the server is failing to start, start looking at the hobbit logs directory. Check here for one location

/var/log/hobbit

Q. After installing the Hobbit client, my msgs tests are "clear" (sometimes refered to as "white")

A. As of the time of this writing, the Hobbit client does NOT have msgs functionality like the BB client does. This can be added by installing the bb-msgs.sh file from the BB client as an external test. Even so, the Hobbit server will turn the test to "clear" instead of the expected status. To correct his issue, you'll have to edit the hobbitlaunch.cfg file (usually found in /etc/hobbit/ or /usr/lib/hobbit/server/etc/) to add --no-clear-msgs to the client channel and restart the server:

CMD hobbitd_channel --channel=client hobbitd_client --no-clear-msgs --log=$BBSERVERLOGS/clientdata.log ...

Q. Tried to down BOARDBUSY: Invalid argument

A.

On Sat, Dec 09, 2006 at 12:08:02PM -0500, Geoff Hallford wrote:
> I am getting the following error message in various Hobbit logs:
>
> 2006-12-04 07:59:46 Tried to down BOARDBUSY: Invalid argument
>
> Does anyone know what this is referring to or what I need to change?

It often shows up when stopping Hobbit - you can ignore it.


Regards,
Henrik

@@ Line 137: / Line 137: @@
 === Install premade hobbit client packages for your OS ===
 * AIX ( http://rpm.razorsedge.org/ )
+* Debian ( http://packages.debian.org/etch-backports/hobbit-client / http://packages.debian.org/lenny/hobbit-plugins )
-* Debian
 * FreeBSD
 * HP-UX ( http://www.razorsedge.org/~mike/software/hobbit/hpux/ )

System Monitoring with Xymon/Administration Guide: Difference between revisions

Revision as of 21:03, 12 February 2008

Architecture of a Hobbit System Monitoring Environment

Picking an OS for Hobbit Server

Linux

Solaris 11/06 U3 on Sun hardware

Pros

Cons

Notes

Hobbit Server: Solaris Intel 11/06 U3 VMware appliance on a 2GB flash pen drive

Hobbit Server and Development: Solaris Intel 11/06 U3 VMware appliance on a 4GB flash pen drive

Hobbit Server Test site

Operational difference between Hobbit and BB BTF

Servers

Clients

References

Capacity Planning

Installation

Install premade hobbit server package for your OS

Install premade hobbit client packages for your OS

Compiling from tar ball

Building from package source using TWW HPMS

Building from src RPM

Hobbit Configuration and tuning

Hobbit(bb) port 1984 encryption

Configure stunnel server to run in hobbit server

Configuring hb client to use port 1999

32 bit vs 64 bit binary for hobbit on Solaris

Configuration

LDAP Authentication

Alerts setting

Tuning

How to shorten the ping test time ?

High Availablity with LinuxHA and DRBD

High Availablity on Solaris

SunCluster

References

FST HA

Hobbit and Remedy Ticket System

Overview

Open Remedy ticket on hobbit alerts

Open Remedy ticket on demand

Migration from BB

Cost (efforts) of Migration

System and Inventory Monitoring

Trouble Shooting Guide

Q. When I click on a status icon I get the message "Status not available". What should I check?

Q. After installing the Hobbit client, my msgs tests are "clear" (sometimes refered to as "white")

Q. Tried to down BOARDBUSY: Invalid argument

Navigation menu

Search