Corrupt inittab + HP UX

Hi Friends,

We faced an issue of "corrupt inittab" after five times reboot of machine HP UNIX due to some Hardware issue. Got an opportunity to work with a Senior.

Error :- "Corrupt inittab"

 With this error,
we first thought that do we have an Ignite backup on tape or not,
Second we though can there be an option if we can restore the inittab from other system and use of the same,
Third we thought of checking from point that if our one disk out of mirror is corrupt or if we need to mirror the same.
Fourth :- If we can boot the machine in single user mode.
Fifth :- If we can boot the machine in LVM maintenance mode in hp ux from hpux prompt using "boot vmunix -lm"
and boot the machine.
a. here after boot up in hpux lvm {lvm maintenance mode}, coming to # prompt.Obviously the experience worked.
He copied the inittab file from /usr/newconfig/etc/inittab to /etc/inittab.
b. we were able to copy the first inittab of the server and then copy it on the /etc/inittab.
c. We reboot the system and checked it worked.
d. after reboot, the vgdisplay command showed correct o/p. we checked the same with previous getsysinfo o/p we had in our /tmp location.
e. checked lvdisplay -v "every root lv".
f. this showed stale devices on root lv on one mirrored disk.
g. here we ran, lvsync /dev/vg00/lvol3
h. it started syncing all stale path hanged into current after some time.
i. we took a reboot to confirm and system came up properly. we compared previous outputs and files. they were fine. we restored some part also with latest backup with us.

It was good learning experience, i wish if i can learn more each day.


Love Sharing
Amit Chopra

Linux System working slow after boot up

Hi Friends,

My Linux box is showing %IO wait 50, when i boot it up. But at the same time, when i take a reboot of it, the % IO wait becomes zero and CPU runs fine.

What all can be the issues? please suggest.


Love Sharing
Amit Chopra

HP UX Interview questions Part 5th

Hi Friends,

Got some good questions, which in an interview are most probably asked:-

1. How to replace a root disk, if corrupted or went bad? write down the steps.
2. How to replace a LOCK disk from a running cluster, what are the steps?
3. If all Nodes are in halt state, then what will be the step to remove and add a new lock disk?
4. What can be the minimum and maximum size of a cluster lock disk?
5. Can we reduce an LV without umount in RHEL?
6.  How to extend an LV in Clustered server?
7. What is selinux in HP UX? How can we implement security in HP UX ?
8. Configuring Bonding in HP UX ?
9. how can we make our system secure?
10. What are the ways using which we can increase our booting process faster?

Love Sharing
Amit Chopra

We all have to Save Nature

Hi Friends,

This change in Weather seems an effect of Global Warming, that in the Month of March is having Heavy Rains in plains. Rain is making the Weather cool again and again, it looks like it is delaying the Summers. This is not usual. This change in weather can cause less production of Wheat. Which will directly cause hike in prices of goods and food material. Somewhere i feel it is due to us and we are making the weather and Nature to react in this manner. Nature will try to make its balance and if we do  not support it. then only and only we all living being have to suffer. Like US had heavy snow fall last year i.e. in 2013 due to disturbances in North Pacific (read in some news paper).

"This all shows we are doing very less towards Nature protection, and we are doing very much towards destroying it."

  We all are consuming Natural resources, but not finding alternatives so that those resources can be saved. Even I consider my self I don't know much even how we can save our Nature's Beauty. But listing those Which i have seen some good people doing and learned watching them, I do the following :-

a. I used to switch off my Vehicle at Red light. It is good that we are having timers at our red lights. which helps us to save petrol and save nature as well. saves money as well, "saving money is investing money" Yes this can help a lot when we all come along and do the same practice.
b. Switch off Lights, fans, AC when they are not in use. little contribution if we all do then we can save a lot. doing the same at Office too at my sitting place.
c. Insist my family members to save Electricity and water as much as they can do. Insist on make proper use of Sun light by opening Windows and Doors.

d. Save Rain Water making umbrella or cone shape technique in water tanks.
e. Thinking of implementing Solar Plant at house, it costs around Rupees 35, 000 in total. so that we can save electricity, But waiting too for that, let's Govt. reduce its cost more.
f.  Planting and gardening at Home, tries to make the green home.
g. Save Water by properly tightens the Taps at Bathrooms, Washrooms.
h. Use less water while Shaving, while brushing my teeth.
i. We all try to keep our surroundings near our Home, Office area clean and clear.
j. Try always to through dust in dustbin, instead of throwing it on roads, the same we can try to keep plastic bags, paper bags, Fruit waste, Vegetable waste, Food waste, etc to through it only in Dustbin. This can be kept in mind while driving car, While holiday at Mountains, While roaming outside, while hanging out in market, while going to stadium to play, at each and every place. where we can help area to remain clean and clear.

Yes there can be much, which can be done. need your comments to find more ways towards making our Nature green, clean and healthy for us and our future.

Love Sharing and Love Nature
Amit Chopra

Issues with resetting password of a user and User Management Tips in HP Unix and Red hat linux

Hi Friends,

Today, we faced an issue like, we were resetting the user's password on the server. But every time we were not able to login into the server for some three to four users. the error was "access denied", Yes we had one ID, using that ID we were trying the resetting of password :) .
after trying multiple times even the IDs were locked out, then we unlocked them and reset again. but same access denied message appeared.
then we checked /var/adm/syslog/syslog.log file, there we saw user was getting message of authentication denied with /bin/false. With /bin/false shell the user can be made to never login into the server. and in our case this happened with a tool running at Security Team end. They run a tool which makes server's users login disabled or put the entry in /etc/passwd appending the user's 7th parameter i.e.shell. like it makes the entry :-
#grep ABC /etc/passwd
ABC:x:UID:GID:/home/ABC:System Admimn:/sbin/sh /bin/false
#
then with this the user will not be able to login until we correct it with.

#usermod -s /bin/bash or /sbin/sh ABC
#

after doing this we can login to the server or system.

Things which i have faced for user's access issues, please share with me also if my friends you have. Need to learn more about it.

a. Shell if not properly defined then user faces  an issue in login.
b. if "passwd" after useradd command is not run then user will not be able to login.
c. Account is locked out. unlock the same with :-
#passwd -u ABC

d. Password should have a special character, a number, alphabets CAPS and small combination.

please avoid using easily crackable passwords like :-

a. India@123
b.  Welcome123

etc. , Password should be combination of four things mentioned above.


e. Issue comes in Linux and HP UNIX servers when users do sudo su - and their password has "@" in their password or in root password then the system doesn't accept it and don't allow you to enter complete password.
f. There can be many errors related to AD authentication or NIS or LDAP. So we can discuss those later.
g. You should remember your password, and change the same timely and if its too critical then keep changing the password and weekly or twice a month.
h. Password expire, User expire, Account expires, we can use " chage -l "username" to check this.


There can be and should be many more, will keep sharing and updating. Please share your experience about User's management issues. so that everyone can know and learn.

=====

 #faillog -r

this command is important to be run if we are using trusted system. we can run this command after unlocking or resetting of a user's password.

passwd -u "username"

will unlock the user's password. and to lock it back. we can use below command :-

passwd -l "username"

====

We faced an issue of only one user was not able to login in several other users. I mean to say that all users except one were able to login and we were getting error

"permission denied error on /var/log/btmp"

so checked the permission using :-

ls -ld
getfacl /var/log/btmp

owner should be root and utmp, permission should be  0600 or 0644. Both of these were working. Even after changing the permission that user was not able to login. so we ran faillog -r command after troubleshooting all parameters. 

it worked.

so  before troubleshooting any user related problem, Request to check if the server or system is trusted or not.





Love Sharing
Amit Chopra

Is talking to self good or bad ?

Dear Friends,

Is talking to self good or bad ? What is your view on it. All positive views invited.

Love Sharing
Amit Chopra
We should Guard our mind so that not a single negative thought can enter into it. In terms of IT, All firewall should be active towards this Negative hacker. Keep a constant or strict eye on such hacker and thief, All possible positive hurdle should be kept like Mount Everest in front of Negative thought.
 

How to change hostname in HP UNIX and Redhat Linux

Hi Friends,


Changing hostname is not a big deal in Redhat Linux and HP UX. But before doing this we have to do proper planning. These Includes :-

HP Unix :-

a. Here we need a downtime to change the hostname as permanent. So be planned about the same, with proper Ignite backups and Binary backups of the DB and the DB backup, if running and installed.

b. Doing it for temporary basis, {I would say dont go for it} until you are pretty much sure about the testing.
c. Yes, in HP UNIX

#hostname amitchopra
#hostname
amitchopra
#

or you keep it like :-

# hostname xyz
#
#hostname
xyz
#

this will make the hostname as amitchopra or xyz for new session also and services may get affected due to it. Services like cluster {cmmcld}, services of oracle running on basis of hostname, applications running on the name of hostname of the server.

so be careful while doing this.

Now here is the game, you have to stop running services on your server after proper downtime, fix the same permanently using below command :-

#set_parms hostname

this command will help in changing the hostname as permanent. It will by itself prompt for rebooting the server. so you can reboot the server. or you can ask your application/DB team to give appropriate downtime for reflecting the hostname permanently.

It seems like a drawback of HP UNIX, if we by mistake run hostname command with any flag to get some information using hostname command then it will keep the flag option as server's hostname. example shown below :-

#hostname -i
#
#hostname
-i
#hostname -a
#
#hostname
-a
#

so this can affect running services also and output of commands like "cmviewcl" or related to service guard cluster will give permission denied error to user. new login user will also see the same hostname like -a or -i or xyz.



and

Redhat Linux gives us this opportunity to keep the hostname temporary and do not affect the running services.

here :-

#hostname amitchopra
#

it will keep the hostname as amitchopra for a terminal session only and other logged in user or will be logging in user will not see this effect. That is the best part.

To make it as permanent fix. we need to keep the entry of new hostname  in /etc/sysconfig/networking file. write the desired hostname there. reload the network services after verifying the /etc/sysconfig/network-scripts/ifcfg-eth0/eth1 file, we should have onboot=yes in that file. check the IP, Port, HW ADDR using ifconfig command. verify and then restart the services of network..


then hostname on redhat linux/linux server will change for permanent. Temporary change in Linux will not affect services or users, where the same can make an affect in HP UNIX.


Love Sharing
Amit Chopra

UNIX or Linux Server Deployment document

Hi Friends,

You can consider below document as a rough sketch of Planning and deployment of UNIX/LINUX/HP UX servers. but wholly and solely, all configurations, Deployment of server, every step depends only and only on your Application requirement and Customer requirement. So lets brief it some points. All comments are welcome for editing and adding any new thing in the document.
Below points are some what applicable for Stand alone server.

Server Deployment Plan

Make a list of available resources for server. 

Inventory and Details of Resources available

S. No.

Descriptions

Quantity in Numbers

Size

Type

From Hardware server end only.

a. Inventory and Details of Resources available 

b. Pre Installation tasks

c. Server Installation Task  

d. Post Installation Tasks

Once we are proceeding towards creation of server, we have to take care of tasks to be done prior to installation of OS on the server. Some of the steps are mentioned below and steps can be added as per the requirement. 

Pre Installation tasks

It includes below points:-

1.       Verification of the Part with the specification of BOM.

2.       Verification of the Rack where Server will be mounted in DC. Which includes checking Power Connectivity with Redundant Power Sources, Dual Network Connectivity, Availability of FC cables for the server if required.

Server Installation Task

1.        Mounting the server in the rack.

2.       Connecting Power cables to power source keeping redundancy in place.

3.       Connecting Network cables.

4.       Labelling the server.

Post Installation Task

1.       Switching On the Server.

2.       Checking Server at BIOS.

3.       Verifying all the available and connected devices.

Pre Installation tasks includes in detail picture 

1	Hardware need to be checked thoroughly, whether it has all parts at its place. Checking the Hardware includes:-
a.	How many NICs are there and with redundancy or not?
b.	How many disks slots are there? What kinds of disk are given on the server? What disks it can support? Because using disks we will be configuring our system.
c	Mirroring is to be done at Hardware level, if yes then there should be a RAID controller card and the same can be configured at booting time. And if the servers are without RAID card then Software RAID will be configured at OS level.
d	How many HBA for FC connectivity are there or not with redundancy?
e	Power Supplies are in redundant or not?
f	DVD or USB device for Installation of OS or using Kick start Linux Installation can be done.
g	At the same time depending upon the requirement and Application requirement, we have to go with 32 or 64 bit of OS on Hardware.
h	If the OS has to be built as a standalone server then it is fine and if not then we have to divide the same into Virtual Partition and then install the server using assignment of resources on the system.

After Pre Installation, Server installation and Post Installation Tasks, we have to perform installation of OS on the server. So below are some of the steps.

Once we are done with above steps and after verifying all the steps, we can proceed with the installation of the OS on the server.

OS Installation Part

1	From Server OS Point of view, we have following task to do.
a.	What version of Linux RHEL they want and of what bit?
b.	What password for root has to be kept?
c.	Hostname of the server
d.	Configuration of NTP, DNS, Networks including IP, Gateway, Net mask, Routes.
e.	Installation of specified software and binaries given by Application team.
f.	Server Hardening:- Hardware RAID, Mirroring or Software RAID, Kernel Tuning, KDUMP, RPMs which has to given by Application team, Citrix Team, etc.
g.	Network Parameters :- Hostname, IP and VIPs, DNS, NETMASK, Gateways, routes,
h.	Security Parameters: - If need to be disabled or if enabled then IPTABLES need to be updated then we have to mention the same in IPTABLES list.

Once Installation part is done, we can proceed with the configuration of parameters of OS. Below are the steps that should be done to for building a basic LINUX server.

OS Configuration Part

1	File system:- the required FS has to be created whether in LVM or Veritas that need to be decided.
	Below FS if need to be created as a separate file system or a single FS, Below is an example:-
	/boot 200 megs
	/  > 10 gigs
	/usr > 3 gigs
	/var > 3 gigs
	/home remainder of space
	2 gigs
2	How much SWAP they need?
3	User and Group creation. User login policy, root login disable.
4	Services running on ports i.e. /etc/services.
5	Disabling the services with complete specification.
6	Nsswitch.conf local authentication.
7	NTP
8	MAIL i.e. SMTP
9	Hosts file update.
10	Securities
11	LIMITS /etc/securities/limits.conf (ulimits)
12	The Default INIT level need to be set, so that further reboot can be avoided.
13	Deciding the services or daemon need to run always.
14	Stopping all those daemons which are not needed and should be stopped using chkconfig.
15	Installing the Networker or AVAMAR or RMAN backup client for taking backup of the OS.

The above also includes Performance Tuning Parameters, but that can be done with taking any with taking performance and experience faced on previous server. We can test other kernel tuning performance  parameters on test servers first and then should go the kernel tuning. Deciding the server performance and behavior is a big task. so it should be done with thorough knowledge. Knowledge of requirement and with following proper Linux/HP doc. Configuring the services and parameters, we can take backup of files and configuration done. So that each and every configured file remains safe. In our scenario the Network IPs and Details are given by Network Team so we have to check with them for the Network details.

Network Configuration Parameter

 IPs , Gateways, Routes need to be given by network Team at your infra. 

 All above points are valid for stand alone server and have 

  There are so many things to add in this. will be adding in next blog. thanks for reading. all comments invited.


Love Sharing
Amit Chopra