I am a big fan of Solaris Volume Manager largely due to it’s simplicity and stability, and being easy to recover when something happens. Most of the places I have worked at SVM is mainly used for root mirroring (and Veritas VxVM is used for everything else). Sanjay Nadkarni from the Sun SVM team posted a message to lvm-discuss with an announcement of the availability of SVM internals. For those of you out there who wants to learn about the SVM history and how it works, it’s time to dive in.

Since the new Daylight Saving Time will be in effect in March 2007, lots of people are concerned about if this will affect Symantec NetBackup. The answer is YES. Here is a response from Symantec on the issue:

http://support.veritas.com/docs/286497

Since the start dates for DST vary each year, these steps will need to
be repeated each year to ensure DST occurs on the correct dates.

This should be enough to address the concerns :)

While Solaris Cluster 3.2 was out a few days ago, I guess most people are still running 3.1 like I do. I found this Sun Cluster 3.1 command line cheat sheet while 
poking around today:

http://sunsolve.sun.com/search/document.do?assetkey=1-9-53040-1

I have been seeing a lot of people asking about the Energy Policy Act of 2005 and Sun’s patches for this lately. Here are two links that should provide you with all the necessary information for Sun Solaris:

http://java.sun.com/developer/technicalArticles/Intl/USDST/
http://sunsolve.sun.com/search/document.do?assetkey=1-26-102775-1

I received an alert notifying me that the /opt on one of our Soalris 10 boxes was 100% full:


# df -h /opt
Filesystem size used avail capacity Mounted on
/dev/dsk/c0t0d0s5 32G 8192E 0K 27983662414130% /opt


I was surprised that the numbers on “used” and “capacity” were surprisingly huge. I checked out exactly how much space had been used on /opt:


# du -sh /opt
1.9G /opt


Also there were still plenty of inode capacity:


# df -o i /opt
Filesystem iused ifree %iused Mounted on
/dev/dsk/c0t0d0s5 17543 3989369 0% /opt


lsof output showed that httpd processes were still hanging on to some lingering disk space after the corresponding files were removed:


# /usr/local/bin/lsof -a +L1 /opt
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
httpd 18129 root 24w VREG 32,13 0 0 9701 /opt (/dev/dsk/c0t0d0s5)
httpd 18129 root 30w VREG 32,13 0 0 10099 /opt (/dev/dsk/c0t0d0s5)
httpd 18133 mobimg 24w VREG 32,13 0 0 9701 /opt (/dev/dsk/c0t0d0s5)
httpd 18134 mobimg 24w VREG 32,13 0 0 9701 /opt (/dev/dsk/c0t0d0s5)
...


But the “SIZE/OFF” all showed up as 0. I had some ideas: either restart the httpd processes or fsck /opt. But since this is a production box, I couldn’t try things out without coordinating a downtime. I spent some time on Sunsolve trying to see if this was a known problem or bug but didn’t find anything useful. I might not hit the right keyword (and it turned out that I didn’t).

So as the last resort, I sent a message to Sunmanagers:

http://www.sunmanagers.org/pipermail/sunmanagers/2006-December/042648.html

I got a couple of replies and most of them suggested the inconsistency between df(1M) and du(1M) might be caused by some running processes which were still hanging on to the lingering disk space after the corresponding file(s) was/were removed. A few sulutions were suggested:


1) restart the https processes
2) fsck /opt
3) simply reboot


While one reply pointed right to the heart of the problem: this is a bug of Solaris 10 and the bug id is 1-26-102285-1. The Sun Alert notification is here:

http://sunsolve.sun.com/search/document.do?assetkey=1-26-102285-1

Sunmanagers is always the right place to go if you are desperately seeking help :)

Here is a serial driver for Mac OS X that supports ”break” signal:

http://sourceforge.net/projects/osx-pl2303/

Nice!

# route -p add 192.168.15.0 192.168.15.1

The static routes will be stored in file /etc/inet/static_routes.

Here are some highlights

Automatic text formatting with vim (Vi IMproved) is easy and I really like that. But on most of the Solaris servers I manage, vim is not installed. So whenever I need to modify a file on a system, I have to constantly remind myself to insert a carriage return when the line has more than 72 characters. Actually I don’t have to anymore after I read this Sun INFODOC (you need to logon Sunsolve to read this doc).

Basically add the following line to ~/.exrc:

map ^V^X !}fmt ^M

Then when you edit a file with vi, type CRTL-x to format the current paragraph so every line has only 72 chars or less.

Learning new stuff everyday…

I just got to know another great Solaris patch tool – pca yesterday. Before I knew pca, I had always relied on PatchPro to apply patches during the days when there was no better patch tool for Solaris. PatchPro has been working great for me, but I never liked its slowness and dependency on WBEM. So I tried pca and was very impressed by what it can do and how easy/light it is compared to smpatch of PatchPro.

To see what pca can do, just type “pca -h”:

$ pca -h
Usage: /home/rfang/bin/pca [OPTION] .. [OPERAND] ..

Operands:
  patch group:    missing, installed, all, unbundled, bad
                  Add r, s or rs at the end to list Recommended,
                  Security or Recommended/Security patches only.
  patch ID:       123456, 123456-78
  patch file:     123456-78.zip, 123456-78.tar.Z
  file name:      patchlist.txt
  pattern:        /dtmail/

Options:
  -l              List patches
  -L              List patches, produce HTML output
  -d              Download patches
  -i              Install patches
  -I              Pretend to install patches
  -x              Download patch cross-reference file
  -y              Do not check for updated patch cross-reference file
  -X dir          Set location of patches cross-reference file
  -P dir          Set patch download directory
  -R dir          Set alternative root directory
  -n              Install only patches which do not require a reboot
  -k              Make patchadd not back up files to be patched
  -G              Make patchadd modify packages in the current zone only
  -a              Ask for SunSolve authentication data interactively
  -H              Don't display descriptive headers
  -r id           Display patch README
  -f dir          Read uname/showrev/pkginfo output from files in dir
  -h              Display this help
  -V              Display debug output
  -v              Display version information

To list all the patches that are available for a system, use the “-l” option:

$ pca -l
Using /var/tmp/patchdiag.xref from Aug/07/06
Host: minnie (SunOS 5.9/sparc/sun4u)

Patch  IR   CR RSB Age Synopsis
------ -- - -- --- --- -------------------------------------------------------
111703 03 < 04 --- 141 SunOS 5.9: /usr/ccs/bin/sccs and /usr/ccs/bin/make patch
111711 12 < 16 R--   1 SunOS 5.9: 32-bit Shared library patch for C++
111712 12 < 15 R-- 215 SunOS 5.9: 64-Bit Shared library patch for C++
112540 23 < 26 R-- 270 SunOS 5.9: Expert3D IFB Graphics Patch
112565 13 < 24 ---  11 SunOS 5.9: XVR-1000 GFB Graphics Patch
112601 09 < 10 RS- 174 SunOS 5.9: PGX32 Graphics
112620 07 < 10 --- 902 SunOS 5.9: Elite3D AFB Graphics Patch
112621 09 < 12 --- 330 SunOS 5.9: Creator and Creator3D: FFB Graphics Patch
112622 11 < 19 --- 344 SunOS 5.9: M64 Graphics Patch
112627 -- < 01 --- 907 SunOS 5.9: TCX Graphics Patch
112661 08 < 10 RS-  64 SunOS 5.9: IIIM and X Input & Output Method patch
112764 07 < 09 R-- 116 SunOS 5.9: Sun Quad FastEthernet qfe driver
112771 15 < 32 -S-  57 Motif 1.2.7 and 2.1.1: Runtime library patch for Solaris 9
112785 49 < 54 RS-  42 X11 6.6.1: Xsun patch
112804 01 < 02 --- 600 CDE 1.5: sdtname patch
112807 16 < 17 RS- 238 CDE 1.5: dtlogin patch
112808 07 < 09 RS- 197 CDE1.5: Tooltalk patch
...

The output is very straightforward, the first column is the patch id, the second column is the patch revision number installed on the system, the fourth column is the most currect patch revision number available from Sunsolve, the fifth indicates if the patch is a recommended patch, a security patch or a bad patch.

Option “-d” will download a specified patch to the current directory, “-i” will download and install the specified patch to the system.

$ pca -d 122716
Using /var/tmp/patchdiag.xref from Aug/07/06

Downloading patches to /home/rfang/patch
------------------------------------------------------------------------------
Retrieving 122716-01 (1/1) ... done

Summary: 1 total, 1 successful, 0 failed

To install all the missing patches on a system, just type “pca -i missing”. The two options that I like most are the “-r” and “-f”. To read the README file of a patch, I used to have to download the patch from sunsolve then unzip it and read the content of the README file. But with pca, I just need to use the “-r” option and it will do everything for you:

$ pca -r 117160
Using /var/tmp/patchdiag.xref from Aug/07/06
Patch-ID# 117160-01
Keywords: overly restrictive krtld
Synopsis: SunOS 5.9: /kernel/misc/krtld patch
Date: Aug/25/2004

Install Requirements: Reboot after installation

Solaris Release: 9

SunOS Release: 5.9

Unbundled Product:

Unbundled Release:

Xref: This patch available for x86 as patch 117161

Topic: SunOS 5.9: /kernel/misc/krtld patch
        ***********************************************************
        NOTE: This patch may contain one or more OEM-specific platform ports.
              See the appropriate OEM_NOTES file within the patch for
              information specific to these platforms.
              DO NOT INSTALL this patch on an OEM system if a corresponding
              OEM_NOTES file is not present (or is present, but instructs not
              to install the patch), unless the OEM vendor directs otherwise.
        ***********************************************************

Relevant Architectures: sparc sparc.sun4u

BugId's fixed with this patch: 4983758

Changes incorporated in this version: 4983758

Patches accumulated and obsoleted by this patch:

Patches which conflict with this patch:

Patches required with this patch: 112233-12 or greater

Obsoleted by:

Files included with this patch:

/kernel/misc/krtld
/kernel/misc/sparcv9/krtld

Problem Description:

4983758 Overly restrictive module dependency checking

Patch Installation Instructions:
--------------------------------

--More--(62%)

Patching isn’t a fun job per se. Once in a while (namely quarterly) I would need to do a massive patching on lot of systems. With PatchPro I have to run smpatch on every system that needs to be patches to analyze and download the patches that system needs. But with pca, I can do all these on a central server. Here is how with pca option “-f”: On the target system (say systemA here) run the following commands:

$ uname -a > uname.out
$ showrev -p > showrev.out (or patchadd -p > showrev.out)
$ pkginfo -x > pkginfo.out

then copy these files to a directory on a system (say systemB here) where you have pca handy. On systemA:

$ pca -f /path/to/the/output/files/ -l (or -d to download)

Isn’t that cool?

One more thing, if you want to list all the dependencies of a particular patch, here is how:

$ pcr -I

NOTE: pca does require perl and wget to work. You can always download a pre-compiled copy of wget from here (for SPARC) and here (for x86)