Category Archives: Fedora

Linux text processing tools – Part 1

This is my attempt to get familiar with the various small yet powerful text processing tools in linux. I am also noting down several oneliners that i found elsewhere.

This is NOT written as a tutorial. For details on usage and options check man pages

tr – tanslate or delete characters

Usage: tr [OPTIONS] SET1 [SET2]

Options include -d (delete), -s (squeeze), -c (complement) etc.

tr supports certain sequences like, [:alnum:],[:alpha:],[:blank:],[:lower:],[:upper:] etc and regular expressions for SET

[tony@localhost tmp]$ tr g e
grgat
great   #pressing CTRL+D quits
[tony@localhost tmp]$ echo tgst | tr g e
test   #adding quotes around SETs are recommended
[tony@localhost tmp]$ echo taxt | tr 'xa' 'se'
test   #Note the translation, x to s and a to e
[tony@localhost tmp]$ echo abCDef | tr '[:lower:]' '[:upper:]'
ABCDEF
[tony@localhost tmp]$tr A-Z a-z
abCDe
abcde
[tony@localhost tmp]$ echo test | tr -d 'e'
tst
[tony@localhost tmp]$ echo abc44jk | tr -d  0-9
abcjk
[tony@localhost tmp]$ echo 12d34sS5c | tr -d  a-z
12345
[tony@localhost tmp]$ echo abCDef | tr -d  [:upper:]
abef
[tony@localhost tmp]$ tr -cd '[:alnum:]'   #strip all symbols
[tony@localhost tmp]$ echo ttest | tr -s 't' 
test
[tony@localhost tmp]$ echo tteesttt | tr -s 'et' 
test
[tony@localhost ~]$ tr -s '\n' #strip empty lines. 
[tony@localhost tmp]$ echo test | tr -c 'e' 'g'
geggg 
[tony@localhost tmp]$ tr -cs 'a-zA-Z0-9_' '\n'< source.c
   #extract keywords, identifiers and comments from C source. ofcourse not perfect!
[tony@localhost tmp]$ tr '[:lower:]' '[:upper:]' <small.txt  >big.txt
[tony@localhost tmp]$ tr -d '\r' <windows.txt  >linux.txt
[tony@localhost tmp]$ tr '\r' '\n' <mac.txt  >linux.txt

 

--------------------

nl - number lines

[tony@localhost tmp]$ cat > list.txt
aaa
bbb

ccc
[tony@localhost tmp]$ nl list.txt 
     1	aaa
     2	bbb

     3	ccc
[tony@localhost tmp]$ nl -ba list.txt 
     1	aaa # -b is numbering syle
     2	bbb
     3	
     4	ccc
[tony@localhost tmp]$ nl -i10 -nrz -s:: -v10 -w4 list.txt 
0010::aaa  #-i increment, -s separator, -v start no, -w width, -n is Format
0020::bbb
      
0030::ccc

 

----------------------

wc - print line, word, and byte counts

[tony@localhost tmp]$ wc list.txt list2.txt 
 3  3 12 list.txt   # the counts are line, word, bytes
 4  3 13 list2.txt
 7  6 25 total
[tony@localhost tmp]$ wc -m list.txt
12 list.txt   #character count
[tony@localhost tmp]$ wc -L sample.txt
82 sample.txt   #longest line length
[tony@localhost tmp]$ ls -1 | wc -l
13   #no of files in current directory
[tony@localhost tmp]$ ps -e | wc -l
178   #no of processes running
[tony@localhost tmp]$ cat /etc/group | wc -l
72

 

----------------------

cat - concatenate or write files, text or binary

[tony@localhost tmp]$ cat > list.txt
aaa

bbb
ccc   #press CTRL+D to stop inputting and save file
[tony@localhost tmp]$ cat list.txt 
aaa

bbb
ccc
[tony@localhost tmp]$ cat f1 f2
aaa
bbb
[tony@localhost tmp]$ cat f1 f2 > f3
[tony@localhost tmp]$ cat f3
aaa
bbb
[tony@localhost tmp]$ cat >> f3
ddd   #append input to file
[tony@localhost tmp]$ cat f1 - f2 > f3 
this is what i entered in the middle
[tony@localhost tmp]$ cat f3
aaa
this is what i entered in the middle
bbb
[tony@localhost tmp]$ cat video.001 video.002 video.003 > vid.avi
  # joining split binary files 

 

----------------------------

tac - concatenate or print in reverse (last line first)

[tony@localhost tmp]$ tac > f1
1
2   #press CTRL+D to stop
[tony@localhost tmp]$ tac f1
1
2
[tony@localhost tmp]$ tac vid.avi > vid2.avi ; mv vid2.avi vid.avi #it wont play
[tony@localhost tmp]$ tac vid.avi > vid2.avi ; mv vid2.avi vid.avi # back to playable. 
  #This is like a mini encryption for binary files.

 

----------------------------

rev — reverse lines of a file or files

[tony@localhost tmp]$ echo this is to be reversed | rev
desrever eb ot si siht
[tony@localhost ~]$ rev .bash_profile 
eliforp_hsab. #

snoitcnuf dna sesaila eht teG #
neht ;] crhsab./~ f- [ fi
crhsab./~ .	
if

smargorp putrats dna tnemnorivne cificeps resU #

nib/EMOH$:HTAP$=HTAP

HTAP tropxe

 

----------------------------

cut - remove sections from each line of files

cut can return section based on no: of bytes (-b), characters (-c), or fields (-f) when fields are separated by a delimiter(-d). Default delimiter is tab.

A range must be provided in each case which consists of one of N, N-M, N-(N to last) or -M (first to M)

[tony@localhost tmp]# cut -c 2-4 
abcdef #press CTRL+D to stop inputting
bcd
[tony@localhost tmp]# cut -c 3
abcdef
c
[tony@localhost tmp]$ cut -c 2,4,7
alongtext 
lne
[tony@localhost tmp]# cut -c -2
abcdef
ab
[tony@localhost tmp]# cut -c 2-
abcdef
bcdef
[tony@localhost tmp]$ cut -c 1,6-9,16-
alongtextwithnospaces
atextspaces
[tony@localhost tmp]# cut -f 2- -d ':'
23:34:45:56 # -d specifies delimiter
34:45:56
[tony@localhost tmp]$ cut -f 2
er rt fg wd ji      
er rt fg wd ji   #cut didnt find the delimiter (default is tab) 
   #so returns whole line
[tony@localhost tmp]$ cut -f 2 -s
er rt fg wd ji    #cut wont print as -s flag is used to
   # prevent printing when delimiter not found.
[tony@localhost tmp]$ cut -d: -f1 /etc/passwd >users.txt

 

-----------------------------

Continued on Linux text processing tools - Part 2

Hybrid graphics in Sony VAIO and Fedora 15

This a report on how i solved radeon driver related booting problem and automatically switched OFF the Radeon card of my Sony VAIO C series laptop in Fedora 15. This laptop has a AMD Radeon and Intel combination.
The steps may be somewhat similar for other laptops with ATI/Intel combinations in Fedora.

NOTE: This method disables the AMD Radeon card upon booting. The aim is to increase battery life(about 1.5 hrs in mine) and reduce the heat. I dont use any graphic intensive applications in linux, so the Radeon is better switched OFF. If you want 3D acceleration or looking for a method for hot switching of cards, you have no luck here.

Background
Fedora 15 comes with a open source radeon driver(not catalyst driver) and includes vgaswitcheroo a mechanism to support switch ON/OFF cards and login out/login card switching.

Upon booting, if the kernel detects hybrid graphics, a directory /sys/kernel/debug/vgaswitcheroo will be created with a file named switch.

The following commands are available,
cat switch – Displays the status of the cards.

[root@localhost ~]# cd /sys/kernel/debug/vgaswitcheroo/
[root@localhost vgaswitcheroo]# cat switch
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :Off:0000:01:00.0

IGD – integrated ie Intel, DIS – discrete ie Radeon, Pwr – power ON, Off – power OFF, and + sign indicates which card if active.

echo OFF > switch – Turns OFF the inactive one.
echo ON > switch – Turns ON the card that is OFF
echo DIS[IGD] > switch – Switches the discrete card [or the IGD] active. You need to logout and login for this to take effect.

GNOME 3 doesnt load in my system with discrete card. So switching to discrete is pretty useless at the moment.

Note that vgaswitcheroo is gone if you have installed the proprietary ‘catalyst’ drivers either using RPMFusion or using the binaries provided at the AMD site.

Booting issue
I often had problem while booting with the system hangs at “dracut: starting plymouth daemon“.

I guess this was due to radeon driver. Not sure though.

The trick was to ‘blacklist’ the radeon driver, and load it once the system has finished booting. And then switch OFF the Radeon card. It worked for me.

You have to do this step only if you have problems during booting caused by the radeon driver. Else skip to here.

Add ‘blacklist radeon’ to the /etc/modprobe.d/blacklist.conf. This will cause the radeon driver not to load during boot.

[root@localhost ~]#echo blacklist radeon >> /etc/modprobe.d/blacklist.conf

Backup and then rebuild the initramfs.

[root@localhost ~]# mv /boot/initramfs-$(uname -r).img{,.bak}
[root@localhost ~]# dracut -f /boot/initramfs-$(uname -r).img $(uname -r)

The dracut command may take a while to finish, and may report some warnings.

If you want, you can reboot now and see whether your issue at boot has resolved.

To load the radeon driver automatically once the boot is fine. We add the script to rc.local.

[root@localhost ~]# echo modprobe radeon >> /etc/rc.local

Note that Radeon is ON after booting whether driver is loaded or not. And radeon driver must be loaded to switch OFF the card,

..

Add the script to switch OFF the Radeon automatically after booting.

[root@localhost ~]# echo "echo OFF > /sys/kernel/debug/vgaswitcheroo/switch" >> /etc/rc.local

Suspend/Hibernate Issue
After a suspend or hibernate, the Radeon card gets switched ON. However, cat /sys/kernel/debug/vgaswitcheroo/switch wrongly shows the Discrete card as OFF. This can be confirmed by checking the remaining battery time or system temperature.

Now to switch the card OFF again, do

[root@localhost ~]# echo ON > /sys/kernel/debug/vgaswitcheroo/switch
[root@localhost ~]# echo OFF > /sys/kernel/debug/vgaswitcheroo/switch

== To Do: automate this ==

To confirm whether the Radeon is switched OFF
One method is to check the remaining battery time before and after switching OFF. There should be a 1-2hrs difference.

Another method i use, is to check the system temperature by installing lm-sensors

[root@localhost ~]# yum install lm_sensors
[root@localhost ~]# sensors
acpitz-virtual-0
Adapter: Virtual device
temp1:        +42.0°C  (crit = +96.0°C)
temp2:        +42.0°C  (crit = +96.0°C)

radeon-pci-0100
Adapter: PCI adapter
temp1:       -128.0°C

Here, the ‘-‘ value indicates that the card is OFF.
A system temperature > 50C at idle also means Radeon ON in my system.


Thanks to,

http://ubuntuforums.org/showthread.php?t=1744188&page=1

doskey @ http://forums.fedoraforum.org/showpost.php?p=1492290&postcount=11

Tags: vgaswitcheroo howto, switchable graphics in fedora 15, turn off AMD/ATI, Hybrid graphics in fedora, switch off dedicated card in fedora, Sony VAIO VPCCB15FG

Accessing the Linux Virtual Console from X

I wanted to copy some text from my Virtual Console (Ctrl + Alt +F[1-6]).
Each of the Virtual Console have a corresponding /dev/vcs[1-6] file.

So doing a cat /dev/vcs2 you get the screen dump of Virtual Console 2.
Note that the output does not contain newline characters, so not properly formatted.

Now the /dev/ttyX files can be used to output text to the corresponding virtual console.

[root@localhost ~]# echo "what is the use of this?" > /dev/tty3

This will print ‘what is the use of this?’ to the virtual console 3

Shrinking and growing a logical volume in Fedora 15

During my Fedora 15 installation, i made two Logical Volumes, HomeLV 50GB for /home and RootLV 5GB for /. This was a mistake and very soon the / was full.

[root@localhost ~]# df -h /
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/FedVG-RootLV
4.8G  4.5G  4.9G  94% /

So i now wanted to shrink the /home and extend the /.

To reduce a logical volume. We first need to reduce the filesystem on it using resize2fs or fsadm. Then use lvresize (or lvreduce) to resize the logical volume.

[root@localhost ~]# resize2fs /dev/mapper/FedVG-HomLV 45G
resize2fs 1.41.14 (22-Dec-2010)
Filesystem at /dev/mapper/FedVG-HomLV is mounted on /home; on-line resizing required
resize2fs: On-line shrinking not supported

Online shrinking is not supported. So now the option is to unmount the underlying filesystem ie, /home. For that i had to logout of the GNOME and work in a Virtual console (Ctrl+Alt+F[1-6]).

Now, a bit more reading of the man page of lvresize and i found that by using -r switch, we can resize the underlying  filesystem  together with the logical volume.

[root@localhost ~]# umount /dev/mapper/FedVG-HomLV
[root@localhost ~]# lvresize -r -L -5G /dev/mapper/FedVG-HomLV
fsck from util-linux2.19.1
dev/mapper/FedVG-HomLV: 900/2949120 files (1.2% non-contiguous), 250200/11796480 blocks

resize2fs 1.41.14 (22-Dec-2010)
Resizing the filesystem on /dev/mapper/FedVG-HomLV to 10977280 (4k) blocks.
The filesystem on /dev/mapper/FedVG-HomLV is now 10977280 blocks long.

Reducing logical volume HomLV to 41.88 GiB
Logical volume HomLV successfully resized

The -L switch with +/-nG directs lvresize it to add or reduce n GB.

[root@localhost ~]# mount /home
[root@localhost ~]# df -h /home
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/FedVG-HomLV
42G  253M   39G   1% /home

 

The next step is to extend the RootLV logical volume and then grow the root filesystem.

[root@localhost ~]# lvextend -l +100%FREE /dev/mapper/FedVG-RootLV
Extending logical volume RootLV to 10.00 GiB
Logical volume RootLV successfully resized

The +100%FREE option directs the lvextend to use the available free extents(free space). Note that the -r switch to resize the underlying filesystem is available here too though i didnt use it.

Grow the filesystem using resizefs. By default, most file system resizing tools will increase the size of the file system to be the size of the underlying logical volume so you do not need to worry about specifying the same size for each of the two commands.

[root@localhost ~]# resize2fs /dev/mapper/FedVG-RootLV
resize2fs 1.41.14 (22-Dec-2010)
Filesystem at /dev/mapper/FedVG-RootLV is mounted on /; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/mapper/FedVG-RootLV to 2621440 (4k) blocks.
The filesystem on /dev/mapper/FedVG-RootLV is now 2621440 blocks long.

Online growing of filesystem is supported although online shrinking not supported as we saw earlier.

[root@localhost ~]# df -h /
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/FedVG-RootLV
9.9G  4.5G  4.9G  48% /

Fedora 15 in SONY VAIO VPC CB 15FG

Model name: Sony VAIO VPCCB15FG

Hardware
Intel Core i5 2410 @ 2.30ghz
Intel HM 65 Express chipset
4GB RAM, 500GB 7200 Toshiba HD
Hybrid graphics AMD Radeon 6630M 1GB DDR3 VRAM / Intel HD 3000
Matshita DVD RW, 15.5” 1920×1080 Full HD WLED screen
ATHEROS WLAN and Ethernet
Realtek Audio, Exmor 1.3MP webcam
HDMI and VGA output. 1 USB 3.0

Installation
Install Media: Fedora 15 64bit DVD

Installation went smooth.
This is the state just after installation,

Sound: Works. Recording: Works.
External monitor via VGA and HDMI: Works in both extended and mirrored fashion. You can use both LCD and external monitor, or LCD ON and external OFF in either mirrored or extended fashion, by using the accelerator buttons. But i couldn’t find a way to switch off the LCD and use only the external monitor.
Accelerator Buttons: Sound+/-, mute, screen brightness+/-, Screen->External monitor, hibernate works. Touchpad On/off and Zoom buttons dont work.
Atheros WLAN: Works. ON/OFF switch: Works.
Atheros Ethernet: Not tested.
Bluetooth: *Not working*. Bluetooth settings show ‘device disabled’
USB 2.0/3.0: Works.
Card Reader: Not tested.
Touchpad: Works fine. Mouse clicks and two finger scrolling works after enabling in the GNOME3 settings.
Webcam: Works.
DVD drive: To Do
Suspend: Works.
Hibernate: Works.

Hybrid graphics: Fedora 15 includes vgaswitchroo module and switching OFF the Radeon card manually works fine. However, when switching to the Radeon from the integrated card, GNOME3 fails to load.

To switch off the Radeon card use the command,

[root@localhost ~]# echo OFF > /sys/kernel/debug/vgaswitcheroo/switch

It can be confirmed whether the card is switched OFF by checking the remaining battery time before and after that command. There will be a difference of ~1.5hrs.

Some issues
Fedora comes with a default Radeon driver. However this seems to have a problem and sometimes during boot the system hangs at “dracut : starting plymouth deamon”. I found a dirty solution read here

As GNOME3 doesnt work with Radeon driver. No 3D support available.

Screen brightness gets reset to maximum after every restart.

..

For those who are interested here is the lspci of the VPCCB15FG

[root@localhost ~]# lspci
00:00.0 Host bridge: Intel Corporation Sandy Bridge DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Sandy Bridge PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation Device 0116 (rev 09)
00:16.0 Communication controller: Intel Corporation Cougar Point HECI Controller #1 (rev 04)
00:1a.0 USB Controller: Intel Corporation Cougar Point USB Enhanced Host Controller #2 (rev 04)
00:1b.0 Audio device: Intel Corporation Cougar Point High Definition Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation Cougar Point PCI Express Root Port 1 (rev b4)
00:1c.1 PCI bridge: Intel Corporation Cougar Point PCI Express Root Port 2 (rev b4)
00:1c.2 PCI bridge: Intel Corporation Cougar Point PCI Express Root Port 3 (rev b4)
00:1c.3 PCI bridge: Intel Corporation Cougar Point PCI Express Root Port 4 (rev b4)
00:1d.0 USB Controller: Intel Corporation Cougar Point USB Enhanced Host Controller #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation Cougar Point LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation Cougar Point 6 port SATA AHCI Controller (rev 04)
00:1f.3 SMBus: Intel Corporation Cougar Point SMBus Controller (rev 04)
01:00.0 VGA compatible controller: ATI Technologies Inc Device 6741 (rev ff)
02:00.0 Network controller: Atheros Communications Inc. AR9285 Wireless Network Adapter (PCI-Express) (rev 01)
03:00.0 SD Host controller: Ricoh Co Ltd Device e823 (rev 04)
03:00.1 System peripheral: Ricoh Co Ltd Device e232 (rev 04)
04:00.0 USB Controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 04)
05:00.0 Ethernet controller: Atheros Communications Device 1083 (rev c0)