Listing threads

Since the introduction of the Native POSIX Threads Library (NPTL) threads have become rather elusive. They don't show up in the default process listing using the ps command and if you are using the ps applet from Busybox they don't show up ever. In this note I will show how to display thread information using the full ps program and how to do the same thing in a shell script if you don't have it.

First, if you do have the luxury of the full ps command (from the procps package) on your target system then all you need to do is add the ā€œ-Lā€ option. For example to list all processes (-A) and all their threads (-L)

	ps -AL

Adding -f (full) gives more information:

	ps -ALf

More interesting, you can create custom output formats using the -o switch followed by a list of keywords. There are over 40 possibilities - read the ps man page for a complete list - among them


  • pid - the process ID
  • tid - the thread ID
  • class - the scheduling class, one of TS (SCHED_OTHER), FF ( SCHED_FIFO) or RR (SCHED_RR)
  • rtprio - priority of real-time tasks (1..99)
  • stat - state, one of R (running) S (sleeping) D (disk wait) T (stopped) Z (zombie)
  • comm - the command name
  • wchan - name of the kernel function in which the process is sleeping

For example

# ps -Leo pid,tid,class,rtprio,stat,comm,wchan
  PID   TID CLS RTPRIO STAT COMMAND         WCHAN
    1     1 TS       - Ss   init            wait
   ...
  277   277 TS       - Ss   sh              wait
 1093  1093 TS       - Sl   thread-test     futex_wait
 1093  1094 TS       - Sl   thread-test     hrtimer_nanosleep
 1093  1095 TS       - Sl   thread-test     hrtimer_nanosleep
 1093  1096 TS       - Sl   thread-test     hrtimer_nanosleep
 1149  1149 TS       - R+   ps              -

Note the four threads in process thread-test each have PID 1093 and have TIDs from 1093 to 1096.

Now, what if you only have the Busybox ps applet, or no ps at all? Of course, you could go and get procps and cross-compile the ps program or maybe find one already cross compiled. For the case where that is not possible or not worth the effort, here is a shell script I use that does almost the same as ps using just basic ash or bash syntax. It gathers information from the /proc file system (as the ps program does) and formats it in a fairly crude way. It is fairly easy to edit to change the information being displayed, as explained in the comments.

#!/bin/sh

# A (b)ash script replacement for ps showing threads and other helpful stuff.
# Especially useful for Busybox based systems which has a very simple ps applet
#
# License: GPLv2
# Copyright 2009 Chris Simmonds, 2net Ltd
# chris@2net.co.uk

show_details ()
{
	# Read the stat entry for this process/thread. See man 5 proc for description
	read _PID _COMM _STATE _PPID _PGRP _SESSION _TTY_NR _TPGID _FLAGS _MINFLT _CMINFLT _MAJFLT \
	     _CMAJFLT _UTIME _STIME _CUTIME _CSTIME _PRIORITY _NICE _NUM_THREADS _IRETVALUE \
	     _STARTTIME _VSIZE _RSS _RSSLIM _STARTCODE _ENDCODE _STARTSTACK _KSTKESP _KSTKEIP \
	     _SIGNAL _BLOCKED _SIGIGNORE _SIGCATCH _WCHAN _NSWAP _CNSWAP _EXIT_SIGNAL _PROCESSOR \
	     _RT_PRIORITY _POLICY _JUNK < /proc/$PID_TO_SHOW/stat

	# Decode policy and priority
	case $_POLICY in
	"0")
		POLICY="TS"
		RTPRIO="- "
		;;
	"1")
		POLICY="FF"
		RTPRIO=$_RT_PRIORITY
		;;
	"2")
		POLICY="RR"
		RTPRIO=$_RT_PRIORITY
		;;
	*)
		POLICY="??"
		RTPRIO="- "
	esac
	# _WCHAN is the address of the kernel function the task is blocked
	# in: not much use, so read /proc/NN/wchan to get the name of the function
	read _WCHAN < /proc/$PID_TO_SHOW/wchan

	# The output is more or less equivalent to "ps -Leo pid,tid,class,rtprio,stat,comm,wchan"
	echo -e "$PID\t$TID\t$POLICY\t$RTPRIO\t$_STATE\t$_COMM\t$_WCHAN"

	# Of course, you can easily change the line above to output more or less information
	# Here is an example which adds nice, vsize and rss
	# echo -e "$PID\t$TID\t$POLICY\t$RTPRIO\t$_NICE\t$_STATE\t$_VSIZE\t$_RSS\t$_COMM\t$_WCHAN"

}

# Print banner. If you change show_details() to output more (or different)
# information, change this as well so you know what is what
echo -e "PID\tTID\tCLS\tRTPRIO\tSTAT\tCOMMAND\tWCHAN"
# echo -e "PID\tTID\tCLS\tRTPRIO\tNICE\tSTAT\tVSIZE\tRSS\tCOMMAND\tWCHAN"

# Get a list of processes from /proc

# Doing it like this gives a numerical listing. The simpler "for p in /proc/[0-9]*"
# would give an alphabetic list in which 2 comes after 10 rather than before.
# N.B. Assumes PIDs are 5 digits or fewer (check /proc/sys/kernel/pid_max)
for p in /proc/[0-9] /proc/[0-9][0-9] /proc/[0-9][0-9][0-9] /proc/[0-9][0-9][0-9][0-9] /proc/[0-9][0-9][0-9][0-9][0-9]; do
	PID=$(basename $p)
	if [ -f /proc/$PID/stat ]; then
		TID=$PID
		PID_TO_SHOW=$PID
		show_details

		# Look in /proc/NNNN/task for a list of threads
		for t in $p/task/*; do
			TID=$(basename $t)
			if [ $TID != $PID ]; then
				PID_TO_SHOW=$TID
				show_details
			fi
		done
	fi
done

Example:

# list_threads.sh 
PID     TID     CLS     RTPRIO  STAT    COMMAND WCHAN
1       1       TS      -       S       (init)  do_wait
...
277     277     TS      -       S       (sh)    do_wait
1093    1093    TS      -       S       (thread-test)   futex_wait
1093    1094    TS      -       S       (thread-test)   hrtimer_nanosleep
1093    1095    TS      -       S       (thread-test)   hrtimer_nanosleep
1093    1096    TS      -       S       (thread-test)   hrtimer_nanosleep
1098    1098    TS      -       R       (list_threads.sh)       0

Finally a note about 'C' libraries: NPTL was incorporated into the GNU C library 2.3 and requires a 2.6 kernel. Many embedded systems use the smaller uClibc library, which doesn't have NPTL, using the older Linux Threads library for POSIX threading. There is an on-going project to integrate NPTL into uClibc but it is not in general use yet. The main difference between NPTL and Linux Threads, so far as this discussion goes, is that in the former all threads have the PID of the process that contains them, whereas with Linux Threads each thread has its own PID. Thus threads show up in process listings by default if you are using Linux Threads but not with NPTL. However, my list_threads script is still useful (I believe) because it can show more information than Busybox.

Comments

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Youtube and Vimeo video links are automatically converted into embedded videos.
  • Lines and paragraphs break automatically.
  • Web page addresses and e-mail addresses turn into links automatically.
  • You may post code using <code>...</code> (generic) or <?php ... ?> (highlighted PHP) tags.

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.