POSIX Threads: Semi-FAQ Revision 5.2

(c) 2001-2006 Michael M. Lampkin
email: michael.lampkin<at>ieee.org


Table of Contents

1. Introduction

  1.1 General Information

  1.2 How to Read This FAQ

  1.3 Coding Standards Used

  1.4 Reader Contributions and Comments

  1.5 Acknowledgments

  1.6 Notice of Copyright and Permissions



2. POSIX

  2.1 A VERY Short History of POSIX

  2.2 How can I determine if my system implements POSIX?

  2.3 Why does my system define _POSIX_VERSION to an unusual value?

  2.4 How do I indicate my code wants only POSIX interfaces exposed?



3. THREADS

  3.1 What is a thread?

  3.2 What is the difference between a Process and Thread?

  3.3 What is a CPU bound thread?

  3.4 What is an IO bound thread?

  3.5 What are advantages of threaded programming?

  3.6 What are disadvantages of threaded programming?

  3.7 When should I make an application threaded?



4. POSIX THREADS / PTHREADS

  4.1 Determining if a system implements POSIX threads?

  4.2 How do I compile a threaded program?

  4.3 How do I define a pthread?

  4.4 How do you start a thread?

  4.5 How do I exit / stop a thread?

  4.6 How many threads can exist simultaneously?

  4.7 Why you normally cannot normally create PTHREAD_THREAD_MAX threads

  4.8 What is a thread attribute?

  4.9 How do I create a variable for thread attributes?

  4.10 How do I destroy a thread attribute variable?

  4.11 How do I identify a specific thread?

  4.12 How do I compare two threads?

  4.13 What is a "joinable" thread?

  4.14 How do you create a "joinable" thread?

  4.15 How do you make an existing thread joinable?

  4.16 How do I join a "joinable" thread?

  4.17 What is a "detached" thread?

  4.18 How do you create a "detached" thread?

  4.19 How do you make an existing thread detached?

  4.20 How do you tell if an existing thread is joinable or detached?

  4.21 How do I tell if a thread is still executing and not terminated?

  4.22 What are pthread_getconcurrency( ) and pthread_setconcurrency?

  4.23 How do I "force" a CPU bound thread to give up the processor?

  4.24 Why does threaded code sometimes segfault when using sleep( )?

  4.25 Can I use SIGALRM with pthreads?

  4.26 How do I modify the priority of a thread?

  4.27 Why is my thread's priority being modified by the system?

  4.28 What is a "race condition"?

  4.29 Can I use a C++ class function to start a thread?

  4.99 Common problems and solutions.



5. PTHREAD ATTRIBUTES ( CONT. )

  5.1 What is the thread Stack (Address and Size) Attribute(s)?

  5.2 How can I determine if a system supports the Stack Attribute(s)?

  5.3 How do you get/set the thread Stack Address and Size Attribute(s)?

  5.4 How do I query an existing thread for its Stack Attribute(s)?

  5.5 How do I modify the Stack Attribute(s) of an existing thread?

  5.6 Why does my program crash / segfault when using Stack Attributes?

  5.7 Can I use the pthread_attr_setstackaddr() function instead?

  5.8 Can I use the pthread_attr_setstacksize() function instead?

  5.9 Can I re-use an attribute object after setting the Stack Address?

  5.10 What is the thread Guard Size Attribute?

  5.11 How can I determine if a system supports the Guard Size Attribute?

  5.12 How do I get/set the thread Guard Size Attribute?

  5.13 Why doesn't setting both Guard Size AND Stack Attributes work as
        expected?

  5.14 What is the thread Scheduling Policy Attribute?

  5.16 What functions get/set the thread Scheduling Policy Attribute?

  5.17 Scheduling policies in detail

  5.18 What is the SCHED_FIFO scheduling policy?

  5.19 What is the SCHED_OTHER scheduling policy?

  5.20 What is the SCHED_RR scheduling policy?

  5.21 What is the SCHED_SPORADIC scheduling policy?

  5.22 How do I query an existing thread for its Scheduling Policy
        Attribute?

  5.23 How do I modify the Scheduling Policy of an existing thread?

  5.24 What is the thread Scheduling Parameter(s) Attribute?

  5.25 What is the structure format for the Scheduling Parameter(s)
        Attribute?

  5.26 What functions get/set the thread Scheduling Parameter(s)
        Attribute?

  5.27 How do I query an existing thread for its Scheduling Parameter(s)
        Attribute?

  5.28 How do I modify the Scheduling Parameter(s) Attribute of an
        existing thread?

  5.29 What are the valid upper and lower priority values?

  5.30 How do I set a thread's priority using the Scheduling
        Parameter(s) Attribute?

  5.31 What is the thread Contention Scope Attribute?

  5.32 How can I determine if a system supports the Contention Scope
        Attribute?

  5.33 What functions get/set the thread Contention Scope Attribute?

  5.34 When should I set the contention to PTHREAD_PROCESS_SCOPE?

  5.35 When should I set contention to PTHREAD_SYSTEM_SCOPE?


6. POSIX MUTEXES

  6.1 How do I determine if my implementation supports mutexes?

  6.2 How do I create a mutex?

  6.3 How do I destroy a mutex?

  6.4 What is a mutex attribute?

  6.5 How do I create a mutex attribute?

  6.6 How do I destroy a mutex attribute (container)?

  6.7 What are the available mutex attributes?

  6.8 What is the mutex Type Attribute?

  6.9 How can I determine if a system supports the mutex Type
        Attribute?

  6.10 What functions get/set the mutex Type Attribute?

  6.11 How do I query an existing mutex for its Type Attribute?

  6.12 How do I modify the Type Attribute of an existing mutex?

  6.13 What is the mutex Shared Attribute?

  6.14 How can I determine if a system supports the mutex Shared
        Attribute?

  6.15 What functions get/set the mutex Shared Attribute?

  6.16 How do I query an existing mutex for its Shared Attribute?

  6.17 How do I modify the Shared Attribute of an existing mutex?

  6.18 What is the mutex Priority Ceiling Attribute?

  6.19 How can I determine if a system supports the mutex Priority
        Ceiling Attribute?

  6.20 What functions get/set the mutex Priority Ceiling Attribute?

  6.21 How do I query an existing mutex for its Priority Ceiling
        Attribute?

  6.22 How do I modify the Priority Ceiling Attribute of an existing
        mutex?

  6.23 What is the mutex Protocol Attribute?

  6.24 How can I determine if a system supports the mutex Protocol
        Attribute?

  6.25 What functions get/set the mutex Protocol Attribute?

  6.26 How do I query an existing mutex for its Protocol Attribute?

  6.27 How do I modify the Protocol Attribute of an existing mutex?

  6.28 What is deadlock?

  6.29 What is priority inversion (with mutexes)?

  6.30 How many mutex objects can exist at the same time?

  6.31 How do I lock and unlock a mutex?

  6.32 Can I test a mutex to see if it is locked?

  6.33 Can I tell a thread to stop waiting to gain ownership of a mutex
        once it starts waiting?

  6.34 Can I put a time limit on the amount of time a thread waits to
        obtain ownership of a mutex?

  6.35 A thread crashed that owned a mutex, how do I unlock it?

  6.36 Is a read write lock a mutex?

  6.37 Is a spin lock a mutex?

  6.38 Which thread gets mutex ownership if several are queued for it?

  6.39 How do I emulate a recursive mutex?

  6.40 What is thread starvation?

  6.41 Can I use "volatile" instead of a mutex?

  6.42 Are mutexes correct if I usually just read data in the critical section?

  6.43 A simple example of using a mutex


7. POSIX CONDITION VARIABLES

  7.1 How do I determine if my implementation supports condition
       variables?

  7.2 How do I create a condition variable?

  7.3 How do I destroy a condition variable?

  7.4 What is a condition variable attribute?

  7.5 How do I create a condition variable attribute?

  7.6 How do I destroy a condition variable attribute?

  7.7 What are the available condition variable attributes?

  7.8 What is the condition variable Clock Attribute?

  7.9 What is the condition variable Process-Shared Attribute?

  7.10 How can I determine if a system supports the Process-Shared
        Attribute?

  7.11 What functions get/set the Process-Shared Attribute?

  7.12 How do I query an existing condition variable attribute for its
        Process-Shared Attribute?

  7.13 How do I use a condition variable - using pthread_cond_wait( )?

  7.14 How do I do a timed wait on a condition variable?

  7.15 How do I do a timed wait on a condition variable using a
        relative time?

  7.16 Why didn't the timed wait return after I modified the system
        clock?

  7.17 How do I wake one thread suspended on a condition variable?

  7.18 How do I wake ALL threads suspended on a condition variable?

  7.19 What is the thundering herd problem?

  7.18 A small example of using a condition variable


8. PTHREADS & SIGNALS

  8.1 What is a signal set and how do I create one?

  8.2 How can I tell if a signal is in a specific signal set?

  8.3 How do I get the signal mask of a thread?

  8.4 How do I set the signal mask of a thread?

  8.5 Can I use sigprocmask( ) instead of pthread_sigmask( )?

  8.6 What is the signal mask for a newly created thread?

  8.7 Using sigwait( ) to receive and process a signal

  8.8 Using sigwaitinfo( ) to receive and process a signal

  8.9 Using sigtimedwait( ) to receive and process a signal

  8.10 How can I send a signal to a specific thread?

  8.11 What is the "recommended" setup for signal handling with
         threads?

  8.12 Why is the wrong thread receiving a signal?

  8.13 A short example of pthreads and catching signals synchronously



9. POSIX SPIN LOCKS

  9.1 How do I determine if my system supports spinlocks?

  9.2 How do I create a spinlock?

  9.3 How do I destroy a spinlock?

  9.4 How many spinlocks can exist at the same time?

  9.5 How do I lock and unlock a spinlock?

  9.6 Can I test a spinlock to see if it is locked?

  9.7 Can I specify how long a thread will spin on a spinlock?

  9.8 A thread crashed that owned a spinlock, how do I unlock it?

  9.9 Which thread gets spinlock ownership if several are spinning on
        it?

  9.10 When should I use / not use a spinlock?



10. POSIX READ-WRITE LOCKS

  10.1 How do I determine if my system supports read-write locks?

  10.2 How do I create a read-write lock?

  10.3 How many read-write locks can exist at the same time?

  10.4 How do I destroy a read-write lock?

  10.5 What is rwlock attribute?

  10.6 How do I destroy a rwlock attribute?

  10.7 What are the available rwlock attributes?

  10.8 What is the rwlock Process Shared Attribute?

  10.9 How can I determine if a system supports the rwlock Shared
         Attribute?

  10.10 What functions get/set the rwlock Shared Attribute?

  10.11 How do I find out if a rwlock is process private or shared?

  10.12 How do I lock and unlock a read-write lock?

  10.13 How do I request a lock for reading on a rwlock?

  10.14 Can I limit the time waiting for a read lock on a rwlock?

  10.15 Can I check if a rwlock would block a reader thread?

  10.16 How do I request a lock for writing on a rwlock?

  10.17 Can I check if a rwlock would block a writer thread?

  10.18 Can I limit the time waiting for a write lock on a rwlock?

  10.19 A thread crashed that owned a read-write lock, how do I unlock
          it?

  10.20 Why are my reader threads suffering from starvation?

  10.21 When should I use / not use a read-write lock?




1. INTRODUCTION

This "FAQ" is a work in progress which was begun in 2001 and continues
to evolve. There are other "FAQ" documents by other authors and groups
concerning multi-threaded programming and this is not meant to supercede
them in any way. It is merely my own compendium of observations,
questions and answers.

Since I am calling it an FAQ, it does contain answers to many of the
questions most frequently asked as pertains to the use of POSIX Threads
as defined by IEEE Std 1003.1-2001. It does mostly stick to POSIX
Threads and not other 3rd party implementations so it would probably be
unsuitable e.g. for an individual desiring more information on threaded
programming in the Microsoft Windows environment.

This FAQ is part one of a multiple document bundle, where the additional
files consists of various source code examples keyed by name to the FAQ
section identifier to which they refer.



1.1 General Information

The FAQ is available in HTML format on the Internet via the following
HTTP URIs:

http://www.cognitus.net

Further editions will be made available in other formats as requested
and if possible.

The FAQ and related materials will be updated on a regular basis in an
attempt to keep it up to date with the current POSIX Threads
specification and submitted questions, answers and code examples.



1.2 How to Read This FAQ

While it is my hope that those new to POSIX Threads, from now on
referred to as pthreads, programming will find this document useful, it
is intended as much to be a reference to the PThreads specification as
it is intended to be a tutorial. Considering this, the organization is
primarily based upon the various available Pthread components and their
functionality instead of being formatted by submission date or the
complexity of the individual topics.

Some source code is included within the FAQ proper but in other cases
the source code is incomplete and not intended for copying or
compilation without the addition of reader supplied code. More complete
code examples will be provided in a separate down loadable tar file in
the near future.



1.3 Coding Standards Used

This FAQ uses a bit of an unusual coding standard as compared to normal
C programming for the example code. A quick and incomplete overview of
that standard is:

* function return values and names are on their own line
* opening and closing bracketing is place on their own line.
* function parameters are on their own line and indented 2 spaces.
* all conditional statements are code blocked.
* any conditional with conditions is organized over and under if the
    statement would exceed 80 characters.
* etc.

There is a rational behind this standard. Quite simply, this document is
being formatted to fit into a standard 80 columns and by having a lot of
over and under type blocking the actual lines of interest are typically
NOT wrapped. The idea is that this will help with the readability of the
example code.



1.4 Reader Contributions and Comments

Comments, corrections and additions are welcomed and should be emailed
to michael.lampkin<at>ieee.org or posted to the forum for review. All
such correspondence will be dealt with as quickly as I can manage and if
appropriate, integrated into the FAQ with proper credits.

The following are a simple set of rules individuals should follow to
expedite the processing of their contribution:

a) All email submissions should be in plain text format only. Any
submission received in a format other than text will probably not be
reviewed.

b) All submissions, if they are for direct inclusion into the FAQ,
should be proof-read for spelling and grammatical errors by the author
and corrections applied where necessary prior to submission. Apologies
in advance for all the spelling and grammatical errors that currently
exist in the document ;-)

c) All submissions, if they are for intended for direct inclusion into
the FAQ, should be clearly labeled as to the section of the FAQ to which
they relate.

d) Code submissions must be in the C programming language and should be
clearly and completely documented. Please note that I will be creating
and making accessible online a coding standards document to compliment
section 1.3 of this document in the near future. I ask that anyone
making code submissions follow those rules for the sake of uniformity.



1.5 Acknowledgments

As this FAQ continues to evolve, I hope and expect that over time a
large number of people will submit comments, questions, answers and
example code to be added to the body of the FAQ and its related
components.

For the record though, it should be stated that as the maintainer I am
responsible for the addition of such material to the FAQ and as such,
accept all responsibility to the limits specified under section 1.6, for
any inaccuracies, omissions or errors contained within this text.



1.6 Notice of Copyright and Permissions

The document "POSIX Threads: Semi-FAQ Revision 5" and associated support
materials (hereafter referred to as the Articles) are Copyright
2001-2006 by Michael M. Lampkin (hereafter referred to as the Author).

These Articles are subject to the following conditions:

* Complying with all applicable copyright laws is the sole
responsibility of the user. No part of the Articles may be reproduced in
any form or by any means, be they mechanical, recording, or otherwise,
or for any purpose other than personal or educational use without the
express written permission of the Author.

* The Author may have intellectual property rights, including but not
limited to patents, patent applications, copyrights and trademarks
covering the subject matter contained within the Articles. The furnished
Articles do not confer up the recipient(s) any license to such
intellectual property except as expressly provided in any written or
digitally signed license agreement from the Author.

* The Author makes no representation or warranty, expressed or implied,
as to the correctness of information contained within or suitability of
the Articles for any particular purpose. The Author shall not be held
liable for any damages suffered, whether real or imagined, as a result
of using information contained within the Articles, any derivatives or
any referenced external items.




2. POSIX

POSIX is an internal standard published by the IEEE that is loosely
based on UNIX System V and Berkeley Unix that defines the basic
interfaces, services and utilities of compliant operating systems. Its
primary aim is to facilitate the portability of applications across many
systems.



2.1 A VERY Short History of POSIX

POSIX.1 was first release in 1988 and became an international standard
in 1990. A number of subsequent revisions have been made to the initial
specification. The most recent revision is referred to as IEEE Std.
1003.1-2001 is actually a single common revision to IEEE Std
1003.1-1996, IEEE Std 1003.2-1992, and the Base Specifications of The
Open Group Single UNIX Specification, Version 2.

Throughout the remainder of this document the use of the word POSIX will
refer to IEEE Std. 1003.1-2001 unless otherwise specified.



2.2 How can I determine if my system implements POSIX?

The POSIX specification requires that the system header file unistd.h
define the symbolic constant _POSIX_VERSION where the given value
indicates the version of IEEE Std 1003.1 C-language binding segment to
which the implementation conforms.

For the 1990 revision compliance the defined value of _POSIX_VERSION
should be 1.

For the 1995 revision compliance the defined value of _POSIX_VERSION
should be 199506L.

For the 2001 revision compliance the defined value of _POSIX_VERSION
should be 200112L.

In addition to _POSIX_VERSION, it is also possible to use the
sysconf( ) function programmatically at runtime and the _SC_VERSION
name value to determine compliance as is shown below.

#define _POSIX_C_SOURCE 200112L

#include <unistd.h>
#include <stdio.h>

int main( int argv, char ** argc )
{
   long x = sysconf(_SC_VERSION);

   printf("%li\n", x );

   return 0;
}


2.3 Why does my system define _POSIX_VERSION to an unusual value?

There are a number of systems which define _POSIX_VERSION and the
return value of sysconf( _SC_VERSION ) to a seemingly random number,
typically greater than 200112L. While this allows you to discover
if a system supplies POSIX it does nothing to help indicate what
level / revision it supports.

Unfortunately, there is no truly easy way to solve this problem.



2.4 How do I indicate my code wants only POSIX interfaces exposed?

To indicate that your code wants only the POSIX interfaces exposed, and
thus assuring that it is POSIX "compliant", you should include the line
in your header file(s):

#define _POSIX_C_SOURCE 1       /* For 1990 revision compliance */
#define _POSIX_C_SOURCE 199506L /* For 1995 revision compliance */
#define _POSIX_C_SOURCE 200112L /* For 2001 revision compliance */

There are a few items to note about the above #define:

* The define statement should be placed in a location within your source
code where it will be processed PRIOR to any include directives. If you
do not do this, the compiler and the prototypes from the system headers
will normally just go to their default condition which is include
everything.

* Even though many systems are still at the 1995 revision level, the
headers are normally well built enough to handle the use of the new
value by checking for a value greater than or equal to (as opposed to
just equal to) 199506L; that value being the one for the 1995 revision.

* The given define is the current standard but in older revisions of the
spec, the required define was given as _POSIX_SOURCE 1 which means if
you see things getting exposed to your code that should not be there,
you may want to revert to see if that helps.





3 THREADS

This section covers threads in general, the POSIX view of a thread and
provides a quick comparison between threads and processes and the
advantages of each.


3.1 What is a thread?

A thread is defined by the POSIX standard to be a single flow of control
within a process and the required system information and resource(s) to
support that flow of control. To paraphrase and put it in simpler terms,
the use of threading allows a single application to appear to perform
multiple tasks (execute multiple threads) at the same time.

Notice that I said "appear to perform multiple tasks at the same time".
This is called concurrency, which is when multiple flows of control (or
in our case, multiple threads) are interleaved without intersection on a
single CPU. To a user or programmer, if the granularity of the
interleaving is high enough, the threads give the appearance of
simultaneous execution even though it is really occurring in a serial
manner.

Parallelism on the other hand deals with multiple threads executing at
exactly the same time. While a single CPU normally cannot typically
perform parallel execution, there are many systems on the market with
multiple CPUs and single CPUs with multiple cores which can are easily
capable of such a feat.

So knowing in very basic terms what a thread is, the difference between
concurrent and parallel execution and the fact that POSIX threads are
only required to exhibit concurrency, I should state for the record that
most kernel space implementations of POSIX threads do provide for
parallelism when there is more than one CPU available on a system. Not
only do the implementations provide for parallelism, they provide it in
a completely transparent fashion meaning that if you have an application
that executes without error on a single CPU system then the same
application will execute and take advantage of parallel execution on
multiple CPU systems without requiring modification of its source code.



3.2 What is the difference between a Process and Thread?

The description / definition of a thread in section 3.1 makes a thread
sound a lot like a process. To be honest, there IS a lot of similarity
between the two but there are also many differences once we delve a bit
deeper into how a thread is created.

Considering that, perhaps the easiest way to understand the differences
between a thread and a process is to look at what information has to be
initialized and maintained for a process and a thread when they are
created. The following two lists are admittedly incomplete but hopefully
will convey the intended point.

A child process created by fork( ) has:

   * Its own program counter.
   * Its own stack / stack pointer.
   * Its own copy of the parent's file descriptors.
   * Its own copy of the parent's directory streams.
   * Its own unique process ID.
   * Its own parent process ID.
   * Its own group ID.
   * Its own user ID.
   * Its own memory space containing a copy of the parent's executable
     code, stack, and variables.

A thread created (using pthread_create(...)) has:

   * Its own program counter and related registers.
   * Its own stack / stack pointer.
   * Its own unique thread ID

Again, the above lists are incomplete and are meant only to convey a
point, not be definitive statements of what occurs when either of the
given functions are called.

The point is that the creation of a process means creating a completely
self-sufficient entity and copying over (almost) all the parent code,
data, and so forth into a new segment of memory. Once created, the child
is completely independent of its parent and modification of either one
(in most cases) has no effect on the other. If the parent and child
desire to share information, this results in the need to explicitly open
routes of communication using sockets, pipes, shared memory or similar
mechanisms between the two processes to facilitate it.

On the other hand, a thread is created and exists within the context of
a process. This means that a thread and its parent process share the
same memory, the same variables (following normal visibility rules), the
same file descriptors, and so forth and do not require the use of any
additional code to facilitate communication between all threads within a
process. This also results in the lifetime of a thread being tied
directly to the lifetime of it's parent process, and that when the
process terminates / exits for any reason, all of its contained threads
must also exit.



3.3 What is a CPU bound thread?

Threads which are CPU bound can be thought of as having their execution
speed tied directly to the execution speed of the CPU. For example, a
thread which had the lone task of calculating the value of PI without
performing any IO operations such as writing data to the screen or disk,
waiting for input from a user and so forth, would be considered
completely CPU bound.



3.4 What is an IO bound thread?

Threads which are IO bound are on the opposite end of the spectrum when
compared to threads which are CPU bound. This means that the speed at
which an IO bound thread executes is not tied to the speed of the CPU on
which it is executing and in fact use very few CPU cycles during their
execution.

An example of this type of thread would be one which waited for input
from a user and once received, performed no processing on it. Under
those conditions, almost all of the thread's time will (likely) be spent
waiting on human interaction; during which time the CPU would be idling
or could readily be made available for use by other threads or processes
on the system.

The term IO bound is a bit of a misnomer in many cases. While IO
operations are very commonly what introduces the described behavior, the
truth is that other common function also introduce it. Two quick
examples are the sleep( ) and wait( ) calls. Despite this, I will use
the term IO bound as I find it less confusing than using the terms CPU
bound and non-CPU bound.



3.5 What are advantages of threaded programming?

There are a number of advantages to using a threaded model. Some which
are commonly give are:

* Applications can be divided into multiple tasks that can then
execute in parallel. A common example of this is an application where
one thread is created to initialize, display and handle manipulations of
a user interface while others are responsible for operations specified
via the interface.

* The ability to more effectively handle applications where multiple
paths of (slow) input or output are handled without having to resort to
system specific functions and tricks.

* On systems with multiple processors, a multi-threaded application
will (normally) take advantage of the additional hardware and can result
in greatly improved overall performance.



3.6 What are disadvantages of threaded programming?

Along with the advantages of employing a threaded paradigm come several
distinct disadvantages. A few of these are:

* Threaded applications are notorious for being harder to debug than
their equivalent single process based applications. One reason for this
is their susceptibility to Heisenberg anomalies. Such anomalies can be
induced by the very act of running the application in a debugger which
can then cause a modification of the thread execution sequences and
consequently prevent the error you are attempting to track from
occurring.

* The programming of multi-threaded applications typically requires a
stricter adherence to proper programming disciplines. A primary reason
for this is the simple fact that all threads within an application
execute within the same address space, in particular, that of their
parent process. Keeping this in mind, it is easy to see that the
corruption of any data / memory by a thread has the potential of causing
unpredictable behavior with another thread in the same process.

* If the number of CPU bound threads in a threaded application
greater than the number of processors on the system, the application
will quite often run (significantly) SLOWER than an equivalent process
based application. A threaded design in such a case adds little if
anything that would allow one to rationalize the additional complexity
introduced by using a threaded design.



3.7 When should I make an application threaded?

Two general things I personally look for in an application when trying
to decide if it would be good candidates for threading are:

   * The application must perform some sort of potentially blocking IO
operation, needs to handle multiple such operations at the same time or
can use the time during such an operation to perform some other task. A
common example is a network server which handles simultaneous client
connections.

   * The application is performing a mostly CPU bound operation which
can be divided into multiple non-dependent operations. A common example
is an application performing matrix operations.

Each application is different though so the best advice is to re-read
the short list of advantages and disadvantages in sections 3.5 and 3.6
and then make a case by case evaluation based on them and your own
experience.





4 POSIX THREADS / PTHREADS

In addition to the other various programming interfaces that POSIX
defines, it also provides those related to threaded programming.

This section details the basic methods to discover if a system
implements POSIX Threads, also known as pthreads, and how to create and
execute a simple pthread.


4.1 Determining if a system implements POSIX threads?

Threads are defined as optional by the POSIX standard which means that
once we determine that the OS is POSIX compliant as described in section
2.2, another check must be done to see if threads are implemented. The
determination can be made by looking in the header files unistd.h for
the symbolic constant _POSIX_THREADS and checking it's value.

If _POSIX_THREADS is undefined or has a value of 0 or -1, this indicates
that POSIX threads are not implemented.

If the symbolic constant _POSIX_THREADS is defined to a value greater
than 0, this indicates that POSIX threads are implemented.

In particular, if the value is 200112L, this is an indication that
pthreads are not only implemented but also compliant with the most
recent POSIX specification.

As with determining if a system is POSIX compliant using sysconf( ),
we can also determine if pthreads are implemented using sysconf( ) at
runtime with the name value of _SC_THREADS as shown below:

#define _POSIX_C_SOURCE 200112L

#include <unistd.h>
#include <stdio.h>

int main( int argv, char ** argc )
{
   long x = sysconf(_SC_THREADS);

   printf("%li\n", x );

   return 0;
}


4.2 How do I compile a threaded program?

Compared to a non-threaded program, you only have to add two items to
the compile line and that is an inclusion of the POSIX libraries. The
library name can vary from system to system but compiling on Linux
with gcc the command line would look like:

gcc -lpthread -lrt -D_REENTRANT -o [OUTPUT NAME] [SOURCE NAME]

The -lpthread says to link with the system pthread library while the
-lrt says to include the real time components. Sometimes you will not
need real time components, for example if you don't use POSIX clocks,
and the -lrt can be removed. On the same note, always including it will
NOT hurt your code.

You may define _REENTRANT within a header file or on the command line
as shown and it indicates that you are specifying explicitly that you
want all available re-entrant versions of functions exposed AND that
errno's will be thread specific.

To quickly cover what a re-entrant function is, consider the functions
which use static buffers to hold data. Because two threads calling such
a function at the same time may end up even in the best of situations,
overwriting each others data, alternative versions of those functions
must be provided if they are to be usable by multi-threaded programs.
The "re-entrant" functions can usually be found by looking up the
normal function name with a trailing _r added.

If you define _POSIX_C_SOURCE >= 199506 then _REENTRANT should
automatically be defined for you, but my motto is better safe than
sorry.


Other systems and options:

AIX with their compiler -

Make certain that pthread.h is the FIRST include file in your source
code.

xlc_r7 -o [OUTPUT NAME] [SOURCE NAME] for ANSI level code
cc_r7 -o [OUTPUT NAME] [SOURCE NAME] for Extended level code


Solaris with their compiler -

cc -lpthread -lrt -D_REENTRANT -o [OUTPUT NAME] [SOURCE NAME]

or ...

cc -mt -lpthread -lrt -D_REENTRANT -o [OUTPUT NAME] [SOURCE NAME]




4.3 How do I define a pthread?

The first step in defining code which will execute in its own thread is
to make certain that your system is POSIX compliant and also supplies a
pthread implementation. The methods by which this can be verified have
been previously covered in sections 2.2 and 4.1 respectively.

The next step is including the following line in your code:

#include <pthread.h>

You must also define a start function to serve as the entry point for
the new thread. All start functions must follow the prototype:

void * [START_FUNCTION_NAME] ( void * arg )

A short example of such a function would be:

void * start
(
   void * arg
)
{
   printf("I am a thread!\n");
   return NULL;
}


4.4 How do you start a thread?

If you create a thread start function / entry point as detailed in
section 4.3 and you call it directly from your program, you end up with
nothing but a single single non-threaded process. To start the function
code executing in it's own thread, you need to utilize the
pthread_create( ) function.

The pthread_create( ) function prototype is:

int pthread_create
(
   pthread_t * thread,
   const pthread_attr_t * attr,
   void * (* start_func ) ( void * ),
   void * arg
);

The arguments you need to supply are:

thread: A pointer to a pthread_t variable supplied by the caller within
which will be placed the identifier for the newly created thread if the
call was successful.

attr: A NULL pointer or (constant) pointer to a pthread_attr_t variable
which contains the desired attributes for the newly created thread. The
setting of thread attributes is covered in more detail later in this
section.

start_func: The address of the function which will be called when the
thread is started e.g. our test_thread function previously given in
this section.

arg: A pointer to a void variable which will be passed to the function
start_func when it is called.


If the call to pthread_create( ) is successful it will set the
pthread * thread parameter so that it uniquely identifies the new
thread and return a value of zero.

It is possible for the call to fail and return one of the following
non-zero error codes:

EAGAIN: The system lacked the necessary resources to create another
thread, or the system-imposed limit on the total number of threads in a
process, as defined by PTHREAD_THREADS_MAX, would be exceeded. See
section 4.6 and 4.7 for more information.

EINVAL: The value specified by attr is invalid.

EPERM: The caller does not have appropriate permission to set the
required scheduling parameters or scheduling policy (see the section on
thread scheduling attributes for further details).

An very simple example with no error checking is:

#define _POSIX_C_SOURCE 200112L

#include <stdio.h>
#include <pthread.h>

void * start
(
   void * arg
)
{
   printf( "I am a thread!\n" );

   return NULL;
}

int main( int argc, char ** argv )
{
   pthread_t thread;

   ( void ) pthread_create( & thread, NULL, & start, NULL );

   pthread_join( thread, NULL );

   return 0;
}

You may notice the use of the pthread_join( ) function. This function
and descriptions of joined and unjoined threads are given in another
section of the document.



4.5 How do I exit / stop a thread?

There are four primary ways that you can exit from a thread or cause it
to terminate. Those methods are:

a) The parent process terminates, in which case all process threads are
terminated.

b) The start function specified during the call to pthread_create( )
returns.

c) The thread explicitly calls the function void pthread_exit( void *
retval ).

d) The thread is canceled.


The last item is the only one which typically requires an in depth
explanation. Since cancellation can be a complex topic with a many
dependencies, it will be covered in its own section later in the FAQ.

Though not listed, a thread can simply crash or even be killed if it
receives a normally unblockable signal such as SIGSEGV. Since such an
event is not normally considered proper behavior, it wasn't placed in
the list.



4.6 How many threads can exist simultaneously?

The number of simultaneous threads allowed is defined by the constant
PTHREAD_THREADS_MAX in the limits.h header file.

It may also be retrieved during runtime using a call to sysconf( ) with
a name value of _SC_THREAD_THREADS_MAX as shown below:

#define _POSIX_C_SOURCE 200112L

#include <unistd.h>
#include <stdio.h>

int main( int argc, char ** argv )
{
   long x = sysconf( _SC_THREAD_THREADS_MAX );

   printf( "%li\n", x );

   return 0;
}

In this case the value returned by the sysconf( ) call may be negative
one (-1) instead of the expected positive long value. If a -1 is
returned, this means that there is "technically" no limit to the number
of simultaneous threads.

Even though _PTHREAD_THREADS_MAX or sysconf( _SC_THREAD_THREADS_MAX )
may return an extremely large value, this value indicates only the
maximum number of threads the system could handle with unlimited memory
and resources. The actual number of maximum threads allowed is really
limited by the available system resources and these values should only
be used as a guideline.



4.7 Why you normally cannot normally create PTHREAD_THREAD_MAX threads

The pthread_create( ) function will return EAGAIN if you are attempting
to create more simultaneous threads than allowed by PTHREAD_THREADS_MAX
or if it runs out of resources. Since we are dealing with a situation
where we have not yet hit the max threads limit, lets consider the
following scenarios:

a) An application (or an implementation) that specifies that all
created threads being given a system level scope. In such a situation,
the threads must be handled by the kernel instead of a second stage
scheduler. The maximum count specified by PTHREAD_THREADS_MAX may be
based on the number of simultaneous threads the second stage scheduler
can handle and it may be able to handle more entities than the kernel
scheduler.

b) An application that specifies that all created threads are joinable
(which is, after all, the default). If a thread is joinable then
additional information, and as a result additional memory, must be
maintained by the system for each thread. The added resource consumption
may be causing your application thread max limit to be lower than the
given maximum.

c) An application that specifies a custom stack size, larger than the
default. As with the situation outline in (b), the additional resource
consumption may skew the real maximum to a lower value than the system
indicates.

The point is (again) that the implementation max values are often (very)
optimistic and you may only be able to create that number under 100%
ideal conditions. Because of this you should only use the values as a
guide and nothing more.



4.8 What is a thread attribute?

A thread attribute is nothing more than an indicator of some aspect
concerning the behavior of a thread. The thread attributes specified by
the current POSIX standard are:

Detach Attribute
Guard Size Attribute
Scheduling Parameters Attribute
Scheduling Policy Attribute
Scope Attribute
Thread Stack Address Attribute
Thread Stack Size Attribute

While a few thread attributes may be manipulated directly using a
function which takes a thread id, more often you will probably find
yourself using a pthread_attr_t type variable during thread creation.
Creating and initializing a pthread_attr_t type variable is covered in
the here and in section 5.



4.9 How do I create a variable for thread attributes?

A thread attribute variable is of type pthread_attr_t so one simply
includes lines similar to:

pthread_attr_t [SOME_VARIABLE_NAME];

The attribute variable is NOT yet usable. You must initialize it by
calling the function:

int pthread_attr_init
(
   pthread_attr_t * attr
);

This initialization MUST be performed prior to the pthread_attr_t
variable's first use. This function allocates the necessary resources
for the thread attribute object and fills it's interior fields with the
default attributes used during thread creation.

Since memory may be allocated to a pthread_attr_t type variable, you
must free that memory using the function described in section 4.9 or
risk a memory leak in you're program.



4.10 How do I destroy a thread attribute variable?

When a thread attribute object is initialized, it is (or more precisely,
may have been) allocated system resources. When the attribute object is
no longer needed, these resources should always be freed by performing
the function call:

int pthread_attr_destroy
(
   pthread_attr_t * attr
);

Once the attribute is "destroyed", you must re-initialize via a call to
pthread_attr_init( ) to re-use in any manner. To do otherwise is to ask
for (at a minimum) unpredictable behavior.



4.11 How do I identify a specific thread?

A thread may be identified by calling the pthread_self() function
defined in the pthread.h header.

The pthread_self() function prototype is:

pthread_t pthread_self
(
   void
);

The returned pthread_t will uniquely identify the thread among those
currently in existence in the process and will be equivalent to the
pthread_t object initialized when pthread_create() was called to start
the thread.

Now for a word of warning: there is NO guarantee that a specific thread
id will not be re-used once a thread with that id has finished execution
and is cleaned up. In other words, unless you are careful, it is
possible to call this function and retrieve an id and think you are
dealing with a thread that has actually already exited.



4.12 How do I compare two threads?

I have often seen code which compares threads using their pthread_t
objects directly. This is incorrect since even though many systems will
define pthread_t as a simple numeric value, POSIX does not in fact
specify a required type for pthread_t and says that it could actually be
a structure which would make a direct == comparison invalid.

Instead the function pthread_equal() should be used if comparison of two
pthread_t variables are desired.

The pthread_equal() function is defined in pthread.h and has the
following prototype:

int pthread_equal
(
   pthread_t t1,
   pthread_t t2
);

Where the parameters t1 and t2 are the two VALID pthread_t objects to
compare for equality. If either t1 or t2 are not valid pthread_t objects
then undefined behavior may result on certain systems.

If the t1 and t2 are equal then the function will return a non-zero
value, while if they are not equal a zero (0) will be returned.



4.13 What is a "joinable" thread?

When a thread is joinable, it indicates that other threads may wait for
it to complete execution by making a call to the pthread_join( )
function. Not only that, but any system resources allocated to the
thread during creation will not be freed until some other thread
successfully calls pthread_join( ) on thread or the thread is detached
using attribute settings or by making a call to pthread_detach( ).


4.14 How do you create a "joinable" thread?

Threads are, by default, created in a joinable state.

If you want to make certain that your system is really creating a
joinable thread then you could explicitly set it during thread creation
by using a thread attribute and the function
pthread_attr_setdetachstate.

The function pthread_attr_setdetachstate is defined as:

int pthread_attr_setdetachstate
(
   pthread_attr_t * attr,
   int detachstate
);

Where attr is a previously initialized pthread attribute and the
detachstate parameter should be set to PTHREAD_CREATE_JOINABLE.

A short code example with no error checking is:

#define _POSIX_C_SOURCE 200112L

#include <stdio.h>
#include <pthread.h>

void * start
(
   void * arg
)
{
   printf( "I am a thread!\n" );

   return NULL;
}

int main( int argc, char ** argv )
{
   pthread_t thread;
   pthread_attr_t attr;

   pthread_attr_init( & attr );

   pthread_attr_setdetachstate( & attr, PTHREAD_CREATE_JOINABLE );

   pthread_create( & thread, &attr, & start, NULL );

   pthread_join( thread, NULL );

   return 0;
}


4.15 How do you make an existing thread joinable?

Unfortunately, once a thread has had its state changed from joinable to
anything else you cannot revert it back to a joinable state.

The only other currently valid state is detached which is covered in
another section.


4.16 How do I join a "joinable" thread?

In a prior we stated that the resources allocated to a thread which is
joinable are not released until another thread joins it or it is
detached. If our desire is to join a thread then that is accomplished
using the obviously named function pthread_join.  This function
suspends execution of the calling thread until the target thread
terminates. In the case where the target thread has already terminated,
the function will return immediately.

The pthread_join() function has the following prototype:

int pthread_join
(
   pthread_t thread,
   void ** value_ptr
);

Where the parameters are specified as:

thread: the target thread the calling thread wants to join.

value_ptr: a NULL value or if non-null then the value returned by the
terminating thread's start function or passed by it to pthread_exit()
will be made available here.

It is possible for the pthread_join() call to fail and return one of the
following non-zero error codes:

ESRCH: The given thread parameter is invalid.

EINVAL: The implementation has detected that the value specified by
thread does not refer to a joinable thread. All threads are created as
joinable by default but may be detached through the manipulation of
thread attributes or a call to pthread_detach function.

EDEADLK: A deadlock condition was detected or the value of thread
specifies the same thread as the one which called the function.

Under almost every circumstance you should either join or detach, as
described in the next sections, all threads to make certain that their
allocated memory is freed and that terminated threads are removed from
the current active thread count.



4.17 What is a "detached" thread?

If you don't want to tie up a thread simply to wait for other threads to
exit and don't care about their return value you can put the threads
into a detached state. A thread in a detached state will have its
allocated resources cleaned up automatically without requiring a call by
some other thread to pthread_join.


4.18 How do you create a "detached" thread?

Threads are, by default, created in a joinable state. To force the
creation of a thread in a detached state you must use a thread
attribute and the function pthread_attr_setdetachstate.

The function pthread_attr_setdetachstate is defined as:

int pthread_attr_setdetachstate
(
   pthread_attr_t * attr,
   int detachstate
);

Where attr is a previously initialized pthread attribute and the
detachstate parameter should be set to PTHREAD_CREATE_DETACHED.

A short code example with no error checking is:

#define _POSIX_C_SOURCE 200112L

#include <stdio.h>
#include <pthread.h>
#include <sys/select.h>

void * start
(
   void * arg
)
{
   printf( "I am a thread!\n" );

   return NULL;
}

int main( int argc, char ** argv )
{
   pthread_t thread;
   pthread_attr_t attr;

   struct timeval timeout;

   pthread_attr_init( & attr );

   pthread_attr_setdetachstate( & attr, PTHREAD_CREATE_DETACHED );

   pthread_create( & thread, & attr, & start, NULL );

   timeout.tv_sec = 3;
   timeout.tv_usec = 0;

   select( 0, NULL, NULL, NULL, & timeout );

   return 0;
}

One item of note about the example is that we use the select function
but not for it's normal purpose. Here we use it as a timer to make the
parent do the equivalent of sleep for 3 seconds prior to termination.
Select is used instead of the normal sleep function because POSIX allows
sleep to utilize signals and signals act very differently in threaded
programs.

The reason for performing the short pause is required is that otherwise
the parent may exit prior to the created thread finishing execution. If
this were to happen then the created thread would also be immediately
terminated and never output the intended message.


4.19 How do you make an existing thread detached?

An existing thread is detached using the function pthread_detach. The
prototype for pthread_detach is:

int pthread_detach
(
     pthread_t thread
);

Where the parameters are specified as:

thread: the thread to detach.

It is possible for the pthread_detach() call to fail and return one of
the following non-zero error codes:

ESRCH: The given thread parameter is invalid.

EINVAL: The specified thread does not refer to a joinable thread e.g.
the thread has previously been detached.

Threads may also be created in a detached state using thread attributes
as discussed in the previous section.

Under almost every circumstance you should either detach or join all
threads to make certain that their allocated memory is freed and that
terminated threads are removed from the current active thread count.



4.20 How do you tell if an existing thread is joinable or detached?

There is no way to directly query an existing thread to determine
whether it is joinable or detached even though you may encounter
code such as the following:

int is_detached( )
{
  if ( EDEADLK == pthread_join( pthread_self( ) )
  {
    return 1;
  }

  /* Result of pthread_join was EINVAL == detached thread */

  return 0;
} 

The idea is that if the thread is joinable and tries to join itself,
then the deadlock condition would be indicated by pthread_join( )
call's return value.  There is one slight problem though.  Even though
most pthread implemenations follow this behavior the specification
states only that the call MAY fail with EDEADLK under such conditions
provides no guarantee.

So if you want to use this type of method you should make certain
the target platform exhibits the desired behavior. Also understand that
you may be introducing cross platform compatability issues.

On a more positive note, the detach state of a thread can be reliably
obtained by querying the thread attribute object which was used
during the creation of the thread as long as the attribute has
not been re-used / modified.

The function defined to discover the Detach State is:

int pthread_attr_getdetachstate
(
   const pthread_attr_t * attr,
   int * detachstate
);

Where the current detach state held in the thread attribute object attr
is written to the integer location specified by the argument
detachstate.

On success (a return value of 0), detach state will contain either the
value PTHREAD_CREATE_JOINABLE or PTHREAD_CREATE_DETACHED; both of which
are self explanatory.

A call to this function should never return an error code though
unpredictable behavior should be expected if either supplied pointer is
invalid.



4.21 How do I tell if a thread is still executing and not terminated?

The answer I have seen most often to this question is to perform a call
the function:

int pthread_kill
(
   pthread_t thread,
   int sig
);

Where the parameter thread is given the ID of the thread you want to
check and sig is a value of 0. If a thread with the given ID is alive,
the function will return a 0 and if it is "dead" (no longer exists) it
will return an error value of ESRCH.

There is one issue about using this functionality and that is the fact
that the POSIX specification allows thread IDs to be re-used. This can
result in the predicament where your code is attempting to check a
particular thread (by ID) and the thread has exited but the
pthread_kill( ) function returns valid (a value of 0) because the ID
your code has possession of has already been re-used.

The end result is that it is very hard, if not impossible, to track the
status of a thread in a reliable using just the POSIX threading API. If
your application requires this type of functionality, it may be better
for you to implement a custom mechanism directly within your code.



4.22 What are pthread_getconcurrency( ) and pthread_setconcurrency?

In the simplest terms, these functions allow access to and manipulation
of the level of mapping from upper level threads to lower kernel level
entities; those entities being known as kernel threads, light weight
processes and a variety of other names depending on their actual
implementation.

Without going into detailed explanations or pointing fingers, I will
simply state that some second level scheduler implementations do NOT
create / assign enough lower level entities. The result is that you
may, with the default setup, see behavior where your application's
main( ) thread executes but none of it's child threads is ever
scheduled any CPU time.

If you run into this issue you might try adding the following to your
code with the pthread_setconcurrency( ) function called prior to the
creation of any threads:

/** Defining _XOPEN_SOURCE 600 exposes the concurrency functions **/

#define _XOPEN_SOURCE 600
#define _POSIX_C_SOURCE 200112L

...

pthread_setconcurrency( VALUE_GT_ONE );

There is a pthread_getconcurrency( ) function to retrieve the current
concurrency level.  The problem with retrieving the concurrency level
is that a call to pthread_getconcurrency( ) will return zero unless 
you have previously set the concurrency level.  This means there is
no way to really retrieve the default concurrency level of the system
on which your application is executing.

I have also heard the suggestion that instead of setting the concurrency
level, you could simply set all threads to PTHREAD_SCOPE_SYSTEM level
using pthread attributes. The rationale being that if the thread is
being scheduled at system scope, you should be by-passing the second
level scheduler and concurrency would be assured. The problem with this
suggestion is that the POSIX specification says an implementation must
support process or system scope but not necessarily both, making it
unviable if you want true cross platform compatibility.



4.23 How do I "force" a CPU bound thread to give up the processor?

By giving up the processor, we mean that the currently executed thread
will release it so that for an application with multiple threads, the
next thread in the queue can begin executing. For this functionality,
the following POSIX function prototype is provided in the sched.h header
:

int sched_yield
(
   void
);

Normally the scheduler will take care of the details for you and calling
sched_yield( ) will not be necessary. There are some "special" cases
where you may want to try sprinkling it in your code. One such case, as
the title of this question indicates, might be when you have a thread
which is 100% or even just heavily CPU bound. In a CPU bound thread the
sched_yield( ) call would indicate or force a break to the scheduler
which can be used to switch to another thread, much in the same way that
an IO bound thread provides such visible breaks during blocking IO
calls.


4.24 Why does threaded code sometimes segfault when using sleep( )?

Here are the basic reasons why you might have segfaults or odd behavior
when using calls to the sleep() function inside threaded code:

* The sleep function is allowed to use signals and quite simply, signals
act much differently in threaded programs as compared to multi-threaded
programs.

* If your system uses SIGALRM or similar, they may be a limit to the
total number of pending signals queued of that type e.g. 8. Exceeding
this could cause a crash, a signal to be lost so that a thread is never
woken or other strange behavior.

There are many solutions to this but the one I normally use since it
seems to work on ANY system which also has the BSD networking core
proper or any derivative of it is calling the select function with all
null FD_SETs.

For example in the follow code the call to select( ) would cause the
main( ) code block to sleep for 3 seconds:

#define _POSIX_C_SOURCE 200112L

#include <sys/select.h>

int main( int argc, char ** argv )
{
   struct timeval timeout;

   timeout.tv_sec = 3;
   timeout.tv_usec = 0;

   select( 0, NULL, NULL, NULL, &timeout );

   return 0;
}


This is certainly not the only solution. You could use nanosleep() if
you knew all the systems supported the REALTIME extension just to name
one.


4.25 Can I use SIGALRM with pthreads?

See "4.24 Why does threaded code sometimes segfault when using sleep(
)?" for a related discussion on sleep, it's potential use of signals and
likely results.



4.26 How do I modify the priority of an existing thread?

Another way to handle a heavily CPU bound thread is to modify it's
priority level instead of constantly making calls to the sched_yield( )
function. You can modify the priority of an existing thread if the
system defines _POSIX_THREAD_PRIORITY_SCHEDULING to 200112L in the
unistd.h header file.

The function to set thread priority is:

int pthread_setschedprio
(
   pthread_t thread,
   int prio
);

Where the thread parameter is the ID of the thread who's priority you
wish to modify and the parameter prio is the new priority level.

This function returns 0 on success and:

ESRCH if the ID specified by the thread parameter does not refer to an
existing thread.

EINVAL if the value of the prio parameter is invalid for the scheduling
policy of the specified thread.

ENOTSUP if an attempt was made to set the priority to an unsupported
value.

EPERM if the implementation does not allow the application to modify
the priority to the value specified.

EPERM if the caller does not have the appropriate permission to set the
scheduling policy of the specified thread.


Setting and getting priority values during thread creation is covered
in the pthread attributes section.



4.27 Why is my thread's priority being modified by the system?

It is a common situation where you will set the priority of a thread
only to notice that is no longer at the specified value after a period
of time. The reason for this is that many operating systems and thread
implementations actually modify the priority of a thread during its
lifetime, with IO bound threads slowly getting their priority raised and
CPU bound threads getting their priority lowered.

This may not make sense at first glance but consider that a truly IO
bound thread is using very little CPU time BUT quite often wants its
operations completed quickly when there is data available for receipt or
transmission. One such case would be an interactive program with one
thread handling user IO such as keyboard entry; we do not want the to
hazard the risk that the IO bound process has too low of a priority and
it taking seconds (or longer) for the data to be processed. To avoid
such a risk of partial "starvation", IO bound threads (and processes for
that matter) will often have their priorities raised over a period of
time. Note that as long as the IO is sporadic, this automatic raising of
priority will not normally degrade the execution of the CPU bound
threads.


4.28 What is a "race condition"?

Probably the easiest way to explain race conditions is to give a very
simple example of one. Imagine you have a threaded program with the
following function:

static int i = 0;

void some_func( )
{
   ++ i;

   if ( 2 == i )
   {
     printf( "some message..." );
   }
}

Now take two threads, we will call them A and B, entering the function
and executing in the following sequence:

* Thread A increments i by 1 so the value of i is now 1
* Thread B increments i by 1 so the value of i is now 2
* Thread B enters the conditional statement and finds that i is now 2
* Thread B prints out the message.
* Thread A enters the conditional statement and finds that i is now 2
* Thread A prints out the message.

If we are used to single threaded programming that called the function
twice we would expect one and only one message to be output to the user.
Instead when using threading we see that the potential exists for 2 ( or
more ) messages to be output. This unexpected behavior when using
threads would be considered a race condition.

Race conditions like the one shown above can happen because threads
execute in an indeterminate manner and for all intent and purpose
"simultaneously". The solution is to use proper synchronization on code
where obtaining the proper result from an operation(s) requires that it
be performed by a single thread at a time.

Synchronization and exclusive access methods are covered in more detail
in the section on pthread mutexes.


4.29 Can I use a C++ class function to start a thread?

Yes you can since, by either planning or happy coincidence, the internal
signature of a static class function is ( usually ) the same as the
signature of a normal C function. So that does result in one restriction
which is that the class function you are going to use is required to be
declared as static.

A short example is:

#define _POSIX_C_SOURCE 200112L

#include <stdio.h>
#include <pthread.h>

class Foo
{
   public:

     static void * start( void * arg )
     {
       printf("I am a thread!\n");

       return NULL;
     }

};

int main( int argc, char ** argv )
{
   pthread_t thread;

   ( void ) pthread_create( & thread, NULL, & Foo::start, NULL );

   pthread_join( thread, NULL );

   return 0;
}

Many people feel that "if you can do it, then do it" but my own
thoughts are it is just as easy to use a regular C function for the
thread then have that function create any required classes. The reason
for this is I feel why take the risk that some compiler out there does
not represent a static C++ class function the same as a C function and
breaks my code.



4.99 Common problems and solutions.

Here are a few items to check before you panic:

a) Make certain you are actually calling pthread_create somewhere in
your code and that it is returning a zero (success).

b) During compilation, make certain you are telling the compiler to
include the pthread library, for example: gcc -lpthread source_name.c.
Some systems / compilers will gladly say the code is compiled and
complete when in fact it has replaced the pthread calls with NOP stubs.

c) Remember that when the process (your main( ) function) exits, all
threads created in the process are destroyed. Try inserting a test call
to select() or a similar call in your main function prior to where it
exits. If the thread(s) now execute properly before the program
terminates, make (more) appropriate modifications like doing
pthread_join calls.

d) You system may require explicit setup of thread concurrency or your
threads may be highly CPU bound and require periodic yielding. Try
adding the line:

#define _XOPEN_SOURCE 600

at the top of your source code prior to the inclusion of any headers and
then, at the top of your main( ) function immediately after your
variable declarations add:

( void ) pthread_setconcurrency( int );

Where the argument in is equal to 2 or greater. The first line says to
expose the X/Open extensions to your application, which is where
pthread_setconcurrency is defined. The pthread_setconcurrency function
call itself provides a hint to the system as to the number of underlying
kernel light weight processes your application process requires at a
minimum.

e) Get away from the computer and relax for a bit. It may just be
temporary code blindness that will be cured by taking a break.






5. PTHREAD ATTRIBUTES ( CONT. )

PThread attributes, which are briefly mentioned in section 4 during the
discussion on joinable and detached threads, are covered in more detail
here. These attributes allow you to modify several aspects of a thread's
behavior during the creation of the thread.



5.1 What is the thread Stack (Address and Size) Attribute(s)?

The Stack Attribute(s) allow you to view and specify the start address
of a thread's stack and the number of bytes it will occupy. These
attributes and the ability to manipulate then are provided since it is
realized that at times the system specific default values may be
inadequate.

Prior to attempting a manipulation of the stack address you should
verify that _POSIX_THREAD_ATTR_STACKADDR and 
_POSIX_THREAD_ATTR_STACKSIZE are defined in unistd.h to 200112L. If
either is defined to a value less than or equal to zero then the
indicated functionality is not provided.


While unusual, there are a few plausible reasons to use these
attributes:

a) A thread which creates a very limited number of local variables could
reduce the stack size saving system resources.

b) A thread which creates a large number of local variables could
increase the stack size to avoid the possibility of over running a stack
of the default size.

c) A thread could create a stack with its size set to the default + n
bytes during the debugging stage where the additional n bytes could
provide for "safe" stack overflowing ( see the topics discussing on the
Guard Size Attribute ).

d) A thread could have it's stack address explicitly set so that it
could be passed to other threads for monitoring.

e) A thread could have it's stack address explicitly set, possibly
allowing in some implementations for the re-use of the memory without
requiring the system to perform its own allocation calls.

There are many uses for these attribute(s) but their use can in some
cases limit the portability of your application across platforms and in
many cases similar functionality is provided through other mechanisms.
Unless you are writing threads which are very recursive (and require
additional stack space) or are in the final optimization stage for your
application, its quite often best not to use these unless you know
exactly what you are doing.



5.2 How can I determine if a system supports the Stack Attribute(s)?

If the header file unistd.h defines the symbolic constant
_POSIX_THREAD_ATTR_STACKADDR to a value greater than 0 then the
implementation should support the getting and setting of the Stack
Address Attribute. If it defined to a value of 200112L then the
current specification is supported.

If the header file unistd.h defines the symbolic constant
_POSIX_THREAD_ATTR_STACKSIZE to a value greater than 0, the
implementation should support the getting and setting of the Stack Size
Attribute. If it defined to a value of 200112L then the
current specification is supported.



5.3 How do you get/set the thread Stack Address and Size Attribute(s)?

The Stack Address and Size Attributes of a thread are obtainable by
querying the thread attribute object which was used during the creation
of the thread. As with most thread attributes, there is no way to
directly query an existing thread to obtain a full list of its current
attributes.

The function defined to discover the Stack Address and Size of a thread
attribute object is:


int pthread_attr_getstack
(
   const pthread_attr_t * attr,
   void ** stackaddr,
   size_t * stacksize
);


One point which should be mentioned concerning the get functionality: if
the address attribute of an attribute object has not previously been
set, the value placed in the stackaddr location is not defined by the
standard. The value which appears to be most often used is a 0 or NULL
to indicate that the system is responsible for determining the stack
address location during the call to pthread_create( ).


The function defined to set the Stack Address and Size Attributes of a
thread attribute object is:

int pthread_attr_setstack
(
   pthread_attr_t *attr,
   void * stackaddr,
   size_t stacksize
);


The stackaddr parameter should contain the start location of the
desired stack area and the stack size should indicate the desired size
in bytes. The stack address (start location) should be properly
allocated and aligned for 8 byte data and the given size must be at
least PTHREAD_STACK_MIN bytes. The maximum valid size is system
dependent and not currently obtainable through a POSIX specified
function call.

On success the set operation will return a zero and on error one of the
following error numbers:

EINVAL The value stackaddr does not have proper alignment for use as a
stack, the value of stacksize is less than PTHREAD_STACK_MIN or the
stacksize exceeds an implementation-defined limit.

EACCES The stack page(s) described by the stackaddr and stacksize
parameters are not both readable and writable by the thread.



5.4 How do I query an existing thread for its Stack Attribute(s)?

There is no way to query an existing thread to determine the value of
either its Stack Address or Stack Size other than saving the unmodified
attribute variable used during the thread's creation.



5.5 How do I modify the Stack Attribute(s) of an existing thread?

Once created, there is no way to modify the value of a thread's Stack
Address or Stack Size.



5.6 Why does my program crash / segfault when using Stack Attributes?

There could be many reasons for this but the most common seems to be
performing something along the lines of:

pthread_kill( thread_id, 0 )

and seeing the system indicating the thread as terminated and then
attempting to re-use the stack space allocated to it. Logic might tell
us that this SHOULD work and usually it and other typical strategies
will BUT sometimes it doesn't for a variety of reasons.

While the above may be an unusual circumstance, perhaps the safest
strategies are:

* fully join any thread who has had stack space explicitly allocated and
wait for the join to return completely OR

* for detached threads reallocate a new stack segment etc. on the heap
for EVERY new thread who's stack attributes you are going to set on
creation.

* for detached threads maintain a pool of previously allocated memory
segments and maintain a callback to indicate the thread exit time making
certain not to re-use the memory segment unless n ( problematic
determination ) amount of time has passed since thread exit.



5.7 Can I use the pthread_attr_setstackaddr() function instead?

Previously the stack address was manipulated via a call to
pthread_attr_setstackaddr(...) but the newer pthread_attr_setstack(...)
is now the recommended method.

Additionally, the function call did not allow a parameter to indicate
and was not required to "automatically" know the direction that the
system stack grew in. This means that for systems where the stack grows
upward, the application would have to pass the start address of the
memory block and for those which grew downward it would have to pass the
address + size. In other words, it created the potential for yet more
incompatibilities associated with this functionality.



5.8 Can I use the pthread_attr_setstacksize() function instead?

Because I said so. End of discussion.

Seriously, the new function which sets the stack address and size at the
same time is much better as it provides better value checking and is
more portable. If all you have is the older function then ok...
otherwise the new setstack option if better.



5.9 Can I re-use an attribute object after setting the Stack Address?

Yes, you can re-use the attribute object but you should be aware of the
fact that you cannot re-use it without changing the Stack Address or you
would, on your second call to pthread_create with it, specifying another
thread using the same stack space as a thread previously created with
the attribute object!

The result is that you must reset that section of the attribute to the
default value which indicates the system / implementation is responsible
for determining and allocating memory for the stack BUT the IEEE Std.
does not define this default value!

So there are two things you can do:

a) Completely re-initialize the attribute object by calling
pthread_attr_destroy( ) followed by a call to the function
pthread_attr_init( ).

b) Create your attribute object and while it is in the default state
(before you do ANY modifications to it), call pthread_attr_getstack( )
and store the supplied default stackaddr and stacksize values returned
in a safe location. Then simply use those stored values to reset
attribute object's stack attributes to their default value by calling
pthread_attr_setstack(thread, addr, size).





5.10 What is the thread Guard Size Attribute?

The Guard Size Attribute controls the amount of memory in bytes, with
the default value being specified as PAGE_SIZE in the limits.h header
file, which will be allocated at the overflow end of the created
thread's stack.

In the case where the thread's stack pointer overflows into the guard
buffer, an error will result. The actual resulting error for such an
overflow incident is not explicitly defined by IEEE Std. 1003-2001
though the following casual statement is made:

"If an application overflows into this buffer an error shall result
(possibly in a SIGSEGV signal being delivered to the thread)."

While this attribute may not appear useful at first glance it does, if
nothing else, allow you to preserve the overflow data so that the
culprit can later be identified.

Warnings regarding this attribute:

a) If the Stack Attribute (location or size) is set, the Guard Size
Attribute is effectively ignored in all cases.

b) Use of guard spaces, while potentially making a threaded application
more robust, is also a drain on system resources. Applications which
know their threads never overflow their stacks should save these
resources by specifying guard buffers of 0 (zero) size or 1 x
{PAGE_SIZE} bytes.

c) Calculations for the guard buffer size should correlate to, at a
minimum, the size of the largest data structure which will ever exist on
the stack. Any size lower this than will defeat the purpose of the guard
buffer. i.e. large data structures need large guard buffers.

d) Conforming implementations may internally round up the Guard Size
Attribute's value to a multiple of {PAGE_SIZE}. Even under such
circumstances, a get operation performed on the Guard Size Attribute
will return the value specified in the most recent set operation, NOT
the rounded value.



5.11 How can I determine if a system supports the Guard Size Attribute?

The Guard Size Attribute and associated functions are part of the X/Open
specific so you should check the file unistd.h for the symbolic constant
_XOPEN_VERSION. If defined to a value greater than 0, preferably 600,
the system supports this attribute.



5.12 How do I get/set the thread Guard Size Attribute?

The guard size of a thread is obtainable by querying the thread
attribute object which was used during the creation of the thread. There
is no way to directly query an existing thread to obtain a full list of
its current attributes.

The function defined to get the Guard Size attribute value is:

int pthread_attr_getguardsize
(
   const pthread_attr_t * attr,
   size_t * guardsize
);

Where the current guard size held in the thread attribute object attr is
written to the size_t location specified by the argument guardsize. On
success (a return value of 0), guardsize will contain a value ranging
from 0 to some system dependent value.

The function defined to set the Guard Size attribute value is:

int pthread_attr_setguardsize
(
   pthread_attr_t * attr, size_t guardsize
);

Where the thread attribute object location is specified by the argument
attr and on success of the function call, will contain the value
specified by guardsize.

The valid values for the argument guardsize are the range 0 (no guard
buffer) to some system independent value with the default being equal to
PAGESIZE bytes. If supplied with a guardsize larger than allowed by the
system, an error number of EINVAL will be returned by the function.


5.13 Why doesn't setting both Guard Size AND Stack Attributes work as
expected?

If you set the stack location or size using the stack attribute, the
guard size attribute will be ignored in ALL cases. You will not get an
error when setting any of these in combination as long as the function
parameters are correct but, again, the guard size will be ignored.

The rationale is that if an application is setting the stack location or
size, that it is using reasonable values for those attributes and
should, if necessary, already have allocated it's own "guard buffer"
area in the specified stack.



5.14 What is the thread Scheduling Policy Attribute?

The Scheduling Policy Attribute allows you to specify the method by
which a thread and it's associated priority and other modifiers are
viewed when comparisons are made to determine which thread will next be
allocated execution time on a CPU.

The following are commonly implemented policies and their value names
as given in the sched.h header:

SCHED_OTHER: Another scheduling policy. (required)

SCHED_FIFO: First in-first out (FIFO) scheduling policy. (required)

SCHED_RR: Round robin scheduling policy. (required)

SCHED_SPORADIC: Sporadic server scheduling policy. (optional - exists
only if _POSIX_SPORADIC_SERVER >= 1 in unistd.h)

Each policy is required to be identified by a unique value. Each policy
also to specify a valid priority range (see "What are the upper and
lower valid priority values?") though the valid priority ranges of
differing policies are allowed to overlap.

Additional system specific policies may exist which are not defined by
IEEE Std. 1003-2001 and which, by their very nature, should be deemed
as non-portable.




5.15 How can I determine if a system supports the Scheduling Policy
Attribute?

If the system defines the symbolic constant
_POSIX_THREAD_PRIORITY_SCHEDULING in the file unistd.h to a value
greater than 0, it supports modifying thread scheduling policies.


5.16 What functions get/set the thread Scheduling Policy Attribute?

The Scheduling Policy Attribute of a thread is obtainable by querying
the thread attribute object which was used during the creation of the
thread.

The function to get the Scheduling Policy Attribute value, if defined,
is given as:

int pthread_attr_getschedpolicy
(
   const pthread_attr_t * attr,
   int * policy
);

Where, on success the function will return zero and the location pointed
to by the parameter policy will contain the value of SCHED_OTHER,
SCHED_FIFO, SCHED_RR, SCHED_SPORADIC or a system dependent policy value.


To set the scheduling policy in a thread attribute we use:

int pthread_attr_setschedpolicy
(
   pthread_attr_t * attr,
   int policy
);


Where the parameter attr is the attribute object to modify and policy is
the value of one of the symbolic constants SCHED_OTHER, SCHED_FIFO,
SCHED_RR, SCHED_SPORADIC or a system dependent policy value.

On success the function returns zero.

On error the function will return EINVAL if the given policy value is
not valid and ENOTSUP is an attempt was made to set the attribute to an
unsupported value. Typically I treat both of these errors as equivalent.


5.17 Scheduling policies in detail

Currently I am directly quoting from IEEE Std. 1003.1-2001 but will, at
a later date, attempt to provide "gentler" descriptions for the
policies.

See the following sections:


What is the SCHED_FIFO scheduling policy?
What is the SCHED_OTHER scheduling policy?
What is the SCHED_RR scheduling policy?
What is the SCHED_SPORADIC scheduling policy?


5.18 What is the SCHED_FIFO scheduling policy?

"Threads scheduled under this policy are chosen from a thread list that
is ordered by the time its threads have been on the list without being
executed; generally, the head of the list is the thread that has been on
the list the longest time, and the tail is the thread that has been on
the list the shortest time." IEEE Std. 1003-2001


5.19 What is the SCHED_OTHER scheduling policy?

"Conforming implementations shall include one scheduling policy
identified as SCHED_OTHER (which may execute identically with either the
FIFO or round robin scheduling policy). The effect of scheduling threads
with the SCHED_OTHER policy in a system in which other threads are
executing under SCHED_FIFO, SCHED_RR, or SCHED_SPORADIC is
implementation-defined.

This policy is defined to allow strictly conforming applications to be
able to indicate in a portable manner that they no longer need a
realtime scheduling policy.

For threads executing under this policy, the implementation shall use
only priorities within the range returned by the sched_get_priority_max(
) and sched_get_priority_min( ) functions when SCHED_OTHER is provided
as the parameter." IEEE Std. 1003-2001


5.20 What is the SCHED_RR scheduling policy?

"This policy shall be identical to the SCHED_FIFO policy with the
additional condition that when the implementation detects that a running
thread has been executing as a running thread for a time period of the
length returned by the sched_rr_get_interval( ) function or longer, the
thread shall become the tail of its thread list and the head of that
thread list shall be removed and made a running thread.

The effect of this policy is to ensure that if there are multiple
SCHED_RR threads at the same priority, one of them does not monopolize
the processor. An application should not rely only on the use of
SCHED_RR to ensure application progress among multiple threads if the
application includes threads using the SCHED_FIFO policy at the same or
higher priority levels or SCHED_RR threads at a higher priority level."
IEEE Std. 1003-2001


5.21 What is the SCHED_SPORADIC scheduling policy?

The sporadic server scheduling policy is supported by the system if the
symbolic constant _POSIX_SPORADIC_SERVER or
_POSIX_THREAD_SPORADIC_SERVER are defined to be greater or equal to 1
( though preferably 200112L ) in unistd.h.

Full description to be added later.

In addition to the required member int sched_priority within the
structure sched_param, SCHED_SPORADIC requires that the following also
be contained:

/* Low scheduling priority for sporadic server. */

int sched_ss_low_priority

/* Replenishment period for sporadic server. */

struct timespec sched_ss_repl_period

/* Initial budget for sporadic server. */

struct timespec sched_ss_init_budget

/* Maximum pending replenishments for sporadic server. */

int sched_ss_max_repl



5.22 How do I query an existing thread for its Scheduling Policy
Attribute?

There is also a function which allows us to get the current policy and
scheduling parameters of a thread:

Code:

int pthread_getschedparam
(
   pthread_t thread,
   int * policy,
   struct sched_param * param
);


Remember that each policy has its own valid priority range and if we
modify the policy, we must at a minimum make certain that the value of
param.sched_priority is within the new policy's range. Any attempt to
set the priority to an out of range value will result in an EINVAL being
returned and neither the policy or the scheduling parameters being
modified.

Finally, since this is part of the REALTIME THREADS component you need
to compile or link with the rt library if your system doesn't do this
for you automatically.



5.23 How do I modify the Scheduling Policy of an existing thread?

To allow for the modification of a thread's scheduling policy after
creation, the following function is provided:


int pthread_setschedparam
(
   pthread_t thread,
   int policy,
   const struct sched_param * param
);

Where thread is the id of the thread you which to modify, policy is the
scheduling policy to apply, and param is the scheduling parameter to
apply.

As with getting the scheduling parameters, this is part of the REALTIME
THREADS components. You will need to compile or link with the rt
library if your system doesn't do this for you automatically.


5.24 What is the thread Scheduling Parameter(s) Attribute?

Each thread is controlled by an associated scheduling policy and
priority and the Scheduling Parameter(s) Attribute, in its minimal
required form, allows you to specify the execution priority of a thread
in relation to the thread's scheduling policy. Under some scheduling
policies, the priority may consist of multiple values including desired
priority, upper and lower bound and so forth. In order to contain these
multiple indicators, the Scheduling Parameter Attribute is specified as
a structure (see next section) instead of as a simple data type.

This parameter does not allow viewing or manipulation of a thread's
actual scheduling policy or its contention scope. The methods by which
those attributes may be manipulated are covered later in the FAQ under
the appropriately labeled section(s).



5.25 What is the structure format for the Scheduling Parameter(s)
Attribute?

The definition for the Scheduling Parameters Attribute is:

Code:

struct sched_param
{
   int sched_priority
   ...
};


Where the sched_priority member is required in all cases and the "..."
represents additional members defined by the specific scheduling
policies. There is no requirement as to how additional members must be
defined so they may occur as discrete members or as union.

At this time, only the SCHED_SPORADIC policy specifies any additional
fields (scheduling policies and the SCHED_SPORADIC policy in particular
are covered in more depth later in the FAQ).



5.26 What functions get/set the thread Scheduling Parameter(s)
Attribute?

The Scheduling Parameters of a thread are obtainable by querying the
thread attribute object which was used during the creation of the
thread. As with most thread attributes, there is no way to directly
query an existing thread to obtain a full list of its current
attributes.

The function defined to discover the Scheduling Parameter Attribute is:

int pthread_attr_getschedparam
(
   const pthread_attr_t * attr,
   struct sched_param * param
);


The function defined to set the Scheduling Parameter Attribute of a
thread attribute object is:

int pthread_attr_setschedparam
(
   pthread_attr_t * attr,
   const struct sched_param * param
);


As previously mentioned, the elements of the structure sched_param
depends on what scheduling policies the system provides. In all cases
though, it will contain at a minimum, the integer member field
sched_priority. The valid value ranges for the sched_priority member are
covered in further detail under section 5.19 and the appropriate
scheduling policy sections.

The set function will return an value of 0 on success, EINVAL if param
is not valid and ENOTSUP if an attempt was made to set the attribute (or
a member of it) to an unsupported value.



5.27 How do I query an existing thread for its Scheduling Parameter(s)
Attribute?

As the scheduling policies have covered in another section, and making
the assumption that you desire only to modify the scheduling attribute
parameters, we should point out there is also a get function available
that lets us query a thread (as opposed to its creation attribute) with
the following prototype:


int pthread_getschedparam
(
   pthread_t thread,
   int * policy,
   struct sched_param * param
);

Where the function returns 0 on success and places the current policy
value and schedule parameters into the like named parameters.

The only error that may occur is ESRCH and only under the condition
where an unknown thread is passed to the function.



5.28 How do I modify the Scheduling Parameter(s) Attribute of an
existing thread?

To allow for the modification of a thread's scheduling parameter(s)
after creation, the following function is provided:

int pthread_setschedparam
(
   pthread_t thread,
   int policy,
   const struct sched_param * param
);

Where thread is the id of the thread you which to modify, policy is the
scheduling policy to apply, and param is the scheduling parameter to
apply.

Calls to this function can return any number of error values including:

ESRCH: The thread parameter refers to an unknown thread.

EINVAL: The policy is invalid or unknown OR one of the scheduling
parameters is invalid for the specified policy

EPERM: The caller does not have the appropriate permissions to set
either policy or scheduling parameters for the thread.

ENOTSUP: An attempt was made to set the policy or scheduling parameters
to an unsupported value.



5.29 What are the valid upper and lower priority values?

Defined within the file sched.h are two function calls which allow you
to retrieve the minimum and maximum valid priority values for a
particular policy.

These functions are:

int sched_get_priority_min
(
   int policy
);

and...

int sched_get_priority_max
(
   int policy
);


The function "policy" parameter is normally one of the following values:

SCHED_OTHER Another scheduling policy.
SCHED_FIFO First in-first out (FIFO) scheduling policy.
SCHED_RR Round robin scheduling policy.
SCHED_SPORADIC Sporadic server scheduling policy.

Notes on the policy parameter:

a) The SCHED_OTHER policy is the most commonly seen as the majority of
systems do not adhere 100% to a FIFO or RR mode and are in fact hybrids.

b) The SCHED_SPORADIC policy will only exist if the symbolic constant
_POSIX_SPORADIC_SERVER is defined to be >=1 (should bee 200112L) in
&lt;unistd.h>.

c) Additional system specific policies may exist which are not defined
by IEEE Std. 1003-2001 and which, by their very nature, should be deemed
as non-portable.



5.30 How do I set a thread's priority using the Scheduling Parameter(s)
Attribute?

Since we are only guaranteed that the sched_priority member will be
present in the structure for a Scheduling Parameter Attribute, we will
focus on its modification.

To guarantee we set the priority member of the attribute in a valid
manner, we must perform some range checking on the member. The sequence
would be (similar to):

a) Create and initialize an attribute variable (if not already done).
b) Get the value of the Scheduling Parameter(s) attribute.
c) Get the default scheduling policy.
d) Using the obtained scheduling policy, obtain its max (or min) allowed
priority range using calls to sched_get_priority_min and
sched_get_priority_max.
e) Validate the desired Scheduling Parameter's (struct) priority member
value against the allowed priority range.
f) If validated, set the Scheduling Parameter(s) attribute, otherwise
skip / perform error handling.


5.31 What is the thread Contention Scope Attribute?

The Scope Attribute controls how a thread has its priority applied when
it is contending for processing time on one of the system's CPUs. There
are currently two contention scopes, process level and system level,
currently defined and represented by the symbolic constants
PTHREAD_SCOPE_PROCESS and PTHREAD_SCOPE_SYSTEM respectively.

A thread which has a process level scope will only have its priority
viewed in the context of the process to which it belongs. In simplistic
terms this means that first the thread's parent process must get a time
slice on a CPU based on it's priority and then, and only then, does the
parent process look at the priority levels of its threads which have a
process level scope. Based on the priority levels of its contained
process scope threads the parent process determines which will be
alloted at least some execution time if possible.

A thread with system scope contention has its priority level applied
directly against the priority level of other processes and system scope
threads. For the most part, when it comes to scheduling by the system a
thread with this type of scope is treated the same as a process would
be.

Comparing the two types of scope we find that threads with a process
scope generally take more CPU horsepower to execute since it is a two
stage system. That is, first the kernel level scheduler must decide if
the parent process should get a time slice then a secondary scheduler
must decide which thread(s) in the process get part of that time slice.
This can also make it hard to predict the 'true' priority of the
threads.

On the other hand, system scope threads require only a kernel level call
for their scheduling and since they are scheduled directly against
processes etc. on the system, their execution times and CPU allocation
is much easier to predict.



5.32 How can I determine if a system supports the Contention Scope
Attribute?

If the system defines the symbolic constant
_POSIX_THREAD_PRIORITY_SCHEDULING in the header unistd.h to a value
greater than 0, it supports the Contention Scope Attribute.



5.33 What functions get/set the thread Contention Scope Attribute?

The contention scope of a thread is obtainable by querying the thread
attribute object which was used during the creation of the thread. As
with most thread attributes, there is no way to directly query an
existing thread to obtain a full list of its current attributes.

The function to get the Contention Scope Attribute value, if defined, is
given as:

int pthread_attr_getscope
(
   const pthread_attr_t * attr,
   int * contentionscope
);

On success (a return value of 0), the contentionscope parameter will
contain the value PTHREAD_SCOPE_PROCESS or PTHREAD_SCOPE_SYSTEM.

The function to set the Contention Scope Attribute value, if defined, is
given as:

int pthread_attr_setscope
(
   pthread_attr_t * attr,
   int contentionscope
);

Where the thread attribute object location is specified by the argument
attr and the contentionscope parameter should be either
PTHREAD_SCOPE_PROCESS or PTHREAD_SCOPE_SYSTEM.

It should be noted that while both of these scopes should be defined,
some implementations may not support the use of both for set operations
and you should either explicitly check for their existence prior to
first use or be prepared to see an error returned of ENOTSUP.



5.34 When should I set the contention to PTHREAD_PROCESS_SCOPE?

If you do NOT require a high(er) level of predictability in reference to
the execution of a thread and / or have concerns about the amount of
time consumed by the system due to the kernel level calls required by
system scope contention, it is recommended that you set the thread's
scope to PTHREAD_PROCESS_SCOPE.

The above is a suggestion and not a definite rule.



5.35 When should I set contention to PTHREAD_SYSTEM_SCOPE?

If you require a high(er) level of predictability in reference to the
execution of a thread compared to other processes existing on the
system, it is recommended that you set the thread's scope to
PTHREAD_SYSTEM_SCOPE.

The above is a suggestion and not a definitive rule.





6. POSIX MUTEXES

A mutex is more properly known as a mutual exclusion lock and is a
mechanism, exposed as a variable and associated functions by the POSIX
specification (IEEE Std. 1003-2001), which provides for the
implementation of exclusive access to a critical section.


To clarify consider the following two definitions:

Critical Section: A critical section is any code path which is required
to execute in an atomic manner. The size of such code path(s) are
application dependent and up to the developer to determine.

Exclusive Access: Exclusive access means that only one thread or process
at a time may have access.


Relating this to real life, consider the situation where an
establishment exists that has a single washroom that is capable of
handling only one person at time. Given this scenario, we can imagine
the following actions being taken by a person in the establishment:

a) Check the washroom door and if unlocked, leave.
b) Check the washroom door and if unlocked, enter and lock the door
     unlocking on exit.
c) Check the washroom door and if locked, leave.
d) Check the washroom door and if locked, form or join a queue waiting
     for it to be unlocked.

If our example were actually an application, the individuals would
represent threads, the washroom would be a critical section of code and
the lock on the washroom door would be the mutex.


Now that we have a (very) basic understand of the concept of a mutual
exclusion lock, the remainder of this section will detail how to create
and manipulate a POSIX mutex object and provide some illustrative
examples of their use.



6.1 How do I determine if my implementation supports mutexes?

If your system provides an implementation of POSIX threading then it
will also provide definitions for basic mutex creation and manipulation.

If a system defines the symbolic constant _POSIX_THREADS in the file
unistd.h to a value greater than 0, it provides an implementation of
POSIX threading.


6.2 How do I create a mutex?

A mutex is of type pthread_mutex_t so the first step of creating an
object of this type for use is to include a declaration in your
application source code as follows:

pthread_mutex_t [MUTEX_VARIABLE_NAME];


Even though the mutex object has been declared and will be instantiated
on execution of the given code, it is still not ready for use. Prior to
the first use of the mutex it must be initialized by calling the
function:

int pthread_mutex_init
(
   pthread_mutex_t * mutex,
   const pthread_mutexattr_t * attr
);


Where the mutex parameter specifies the location of the mutex we wish to
initialize and the attr parameter specifies the desired mutex
attributes. If the attr parameter is NULL, the default mutex attributes
are used by the system. See the next few sections for details about the
pthread_mutexattr_t type, its creation and the manipulation of mutex
attributes.

On success the function will return a value of zero, the mutex will be
initialized and in an unlocked state.


It is also possible to declare and initialize a static mutex using the
following:


static pthread_mutex_t [MUTEX_VARIABLE_NAME] =
PTHREAD_MUTEX_INITIALIZER;


The use of PTHREAD_MUTEX_INITIALIZER is the same as calling
pthread_mutex_init( ) with default attributes being provided. The
primary difference between this method of initialization and the
pthread_mutex_init( ) function call is that PTHREAD_MUTEX_INITIALIZER
does not allow you to specify attributes and will never return an error
(even if one were to occur).


6.3 How do I destroy a mutex?

When a mutex is no longer needed it should be destroyed so that any
allocated system resource many be freed for reuse. The function which
provides mutex destruction is:

int pthread_mutex_destroy
(
   pthread_mutex_t * mutex
);

The function will return a zero on success. A value of EBUSY is returned
if the mutex is currently in use and a value of EINVAL if the mutex is
invalid.


6.4 What is a mutex attribute?

A mutex attribute allows you to modify the various attributes of a mutex
(yes, I realize that is a recursive definition) to values other than
those provided by default initialization.

The mutex attribute data type is defined in the file pthread.h and
specified to be pthread_mutexattr_t.


6.5 How do I create a mutex attribute?

The mutex attribute (container) type is declared in the file pthread.h
and defined as pthread_mutexattr_t. The first step in preparing an
object of this type for use is to include a declaration in your
application source code as follows:


pthread_mutexattr_t [MUTEX_VARIABLE_NAME];


As with a mutex object, even though the mutex attribute object has been
declared and will be instantiated on execution of the given code, it is
still not ready for use. This means that prior to the first use of the
mutex attribute it must be initialized by calling the function:

int pthread_mutexattr_init
(
   pthread_mutexattr_t * attr
);


Where the attr parameter specifies the location of the mutex attribute
we want to initialize. On success the function will return a value of
zero and the attribute object will be initialized to the default
system values.

Once created, the various contained attributes may be manipulated
through supplied POSIX function calls. Those functions are are covered
in the next few sections.



6.6 How do I destroy a mutex attribute (container)?

When a mutex attribute object is no longer required, it should be
explicitly destroyed to free up any resources which may have been
allocated to it. The function provided for destruction of mutex
attribute objects is:


int pthread_mutexattr_destroy
(
   pthread_mutexattr_t * attr
);


Failure to destroy mutex attribute objects which are no longer required
may result in memory leakage.


			
6.7 What are the available mutex attributes?

The available mutex attributes are:


Type
Shared
Priority Ceiling
Protocol


Each of these attributes are described in their own appropriated labeled
section of the FAQ.



			
6.8 What is the mutex Type Attribute?

The Type Attribute allows your code to specify how a mutex will react
when an attempt is made by a thread to lock or unlock the mutex; in
particular it specifies how the mutex will react under different
potential error conditions.

The specified Type Attribute values and their descriptions are:

PTHREAD_MUTEX_DEFAULT

An attempt by a thread to recursively lock a mutex of this type, to
unlock a mutex of this type which was not locked by thread or to unlock
a mutex of this type which was not previously locked will result in
undefined behavior.

PTHREAD_MUTEX_NORMAL:

A mutex of this type will not detect deadlock where deadlock is defined
as (but not limited to) a thread attempting to relock a mutex without
first unlocking it. Any attempt to unlock a mutex of this type locked by
another thread or to unlock a mutex which is current unlocked will
result in undefined behavior.

PTHREAD_MUTEX_ERRORCHECK

A mutex of this type will return an error if a thread attempts to lock a
mutex for which it already hold the lock, if a thread attempts to unlock
a mutex which another thread locked or if a thread attempts to unlock an
unlocked mutex.

PTHREAD_MUTEX_RECURSIVE

A mutex of this type may be locked multiple times by the same thread
though an equal number of unlock operations must be performed before
another thread may obtain the mutex. If a thread attempts to unlock a
mutex which another thread locked or an attempt is made to unlock an
unlocked mutex, an error will be returned.

It is suggested that any release level applications (unless there is a
rationale to do otherwise) specify PTHREAD_MUTEX_RECURSIVE for the mutex
type attribute. If the system does not provide the recursive type mutex
then PTHREAD_MUTEX_ERRORCHECK is probably the next safest choice.


			
6.9 How can I determine if a system supports the mutex Type Attribute?

Your application should check that the symbolic constant _XOPEN_VERSION
is defined in unistd.h to a value equal to or greater than 1. The
preferred value is 600.


			
6.10 What functions get/set the mutex Type Attribute?

The Type Attribute of a mutex attribute object can be obtained by
calling the function:


int pthread_mutexattr_gettype
(
   const pthread_mutexattr_t * attr,
   int * type
);


Where the parameter attr refers to the location of a previously declared
and initialized mutex attribute object. On success the function will
return a zero and the location pointed to by the parameter type will
contain mutex attribute object's type attribute.


The Type Attribute of a mutex attribute object can be set by calling the
function:


int pthread_mutexattr_settype
(
   pthread_mutexattr_t * attr,
   int type
);


Where the parameter attr refers to the location of a previously declared
and initialized mutex attribute object and type has been set to
PTHREAD_MUTEX_DEFAULT, PTHREAD_MUTEX_NORMAL, PTHREAD_MUTEX_ERRORCHECK,
or PTHREAD_MUTEX_RECURSIVE.


			
6.11 How do I query an existing mutex for its Type Attribute?

The Type Attribute is exposed to application code only via the mutex
attribute object used to initialize a mutex and cannot be obtained
directly from the mutex or modified once the mutex is created.


			
6.12 How do I modify the Type Attribute of an existing mutex?

The Type Attribute is exposed to application code only via the mutex
attribute object used to initialize a mutex and cannot be obtained
directly from the mutex or modified once the mutex is created.


			
6.13 What is the mutex Shared Attribute?

The default for this attribute is PTHREAD_PROCESS_PRIVATE and is
specified to indicate that only those threads within the process which
created the mutex are allowed to manipulate it.

If the attribute is set to PTHREAD_PROCESS_SHARED, then the mutex may
have operations performed on it by any thread that has access to the
memory where the mutex is allocated, even if the other threads were
created and contained by a process other than the one which created the
mutex.

It a mutex with its shared attribute set to PTHREAD_PROCESS_PRIVATE is
accessed from another process the result of any such operation is
undefined.


			
6.14 How can I determine if a system supports the mutex Shared
Attribute?

If the symbolic constant _POSIX_THREAD_PROCESS_SHARED is defined in
unistd.h to a value greater than zero, the system provides support of
the Shared Attribute.


			
6.15 What functions get/set the mutex Shared Attribute?

To query a mutex attribute variable for the current state of its
(Process) Shared Attribute, use the function:


int pthread_mutexattr_getpshared
(
   const pthread_mutexattr_t * attr,
   int * pshared
);


Where the parameter attr is a pointer to the location of the mutex
attribute object we want to query and pshared is a pointer to the
location where the obtained information (either PTHREAD_PROCESS_PRIVATE
or PTHREAD_PROCESS_SHARED) will be placed.

To set the (Process) Shared Attribute of a mutex attribute variable, the
following function is supplied:


int pthread_mutexattr_setpshared
(
   pthread_mutexattr_t * attr,
   int pshared
);


Where the parameter attr is a pointer to the mutex attribute object you
wish to modify and the parameter pshared indicates the (new) desired
sharing behavior.



6.16 How do I query an existing mutex for its Shared Attribute?

The (Process) Shared Attribute is exposed to application code only via
the mutex attribute object used to initialize a mutex and cannot be
obtained directly from the mutex or modified once the mutex is created


6.17 How do I modify the Shared Attribute of an existing mutex?

The (Process) Shared Attribute is exposed to application code only via
the mutex attribute object used to initialize a mutex and cannot be
obtained directly from the mutex or modified once the mutex is created.



6.18 What is the mutex Priority Ceiling Attribute?

When a mutex is initialized with a Priority Ceiling specified, any
thread obtaining a lock on the mutex will have it's priority compared to
that mutex's Priority Ceiling and:

a) If the thread's priority is lower, its priority will be raised to the
same level as the mutex's Priority Ceiling for the duration of time that
it holds the lock on the mutex.

b) If the thread's priority is equal to or greater than the Priority
Ceiling, it will execute without modification of it's priority.


A benefit of this functionality is that as long as all threads which
access the mutex have a priority less than or equal to the Priority
Ceiling of the mutex, priority inversion will not occur. Priority
inversion is when a thread is (involuntarily) suspended waiting to gain
access to a resource, in this case a lock on the mutex, while a thread
of lower priority continues to execute. Of course, if any thread which
accesses the mutex has a higher priority than the mutex's Priority
Ceiling, priority inversion is still possible and this benefit is lost.

Another use can be see if we consider the scenario (for example) where
we are designing a system which need "real time" responsiveness to data
input. On such a system, all threads may be of the same priority but the
segment of code responsible for reading data may be protected by a mutex
with a Priority Ceiling higher than the thread priorities. This would
mean that any thread entering the protected segment would have its
priority raised and would result in the code executing in a more timely
fashion (that is, less likely to be pre-empted by another thread).


6.19 How can I determine if a system supports the mutex Priority Ceiling
Attribute?

If the symbolic constant _POSIX_THREAD_PRIO_PROTECT is defined in
unistd.h to a value greater than 0, the system supplies support of the
Priority Ceiling Attribute.


6.20 What functions get/set the mutex Priority Ceiling Attribute?

To query a mutex attribute variable for the current value of its
Priority Ceiling Attribute, use the function:

int pthread_mutexattr_getprioceiling
(
   const pthread_mutexattr_t * attr,
   int * prioceiling
);

Where the parameter attr is a pointer to the location of the mutex
attribute object we want to query and prioceiling is a pointer to the
location where the obtained information will be placed.


To set the (Process) Shared Attribute of a mutex attribute variable, the
following function is supplied:

int pthread_mutexattr_setprioceiling
(
   pthread_mutexattr_t * attr,
   int prioceiling
);

Where the parameter attr is a pointer to the mutex attribute object you
wish to modify and the parameter prioceiling indicates the (new) desired
priority ceiling.

The range of valid values for the parameter prioceiling are the same as
the range of priorities (values) allowed by the scheduling policy
SCHED_FIFO. The lower and upper bounds for this policy may be obtained
by calling sched_get_priority_min(SCHED_FIFO) and
sched_get_priority_max(SCHED_FIFO) respectively.


6.21 How do I query an existing mutex for its Priority Ceiling
Attribute?

To query an existing mutex for the current value of its Priority Ceiling
Attribute, use the function:

int pthread_mutex_getprioceiling
(
   const pthread_mutex_t * mutex,
   int * prioceiling
);

Where the parameter mutex is a pointer to the location of the mutex we
want to query and prioceiling is a pointer to the location where the
obtained information will be placed.


6.22 How do I modify the Priority Ceiling Attribute of an existing
mutex?

To set / modify the Priority Ceiling Attribute of an an existing mutex,
use the function:

int pthread_mutexattr_setprioceiling
(
   pthread_mutex_t * mutex,
   int prioceiling
);



Where the parameter mutex is a pointer to the mutex you wish to modify
and the parameter prioceiling indicates the (new) desired priority
ceiling.

The range of valid values for the parameter prioceiling are the same as
the range of priorities values allowed by the scheduling policy
SCHED_FIFO. You can determine the valid lower and upper bounds for this
policy by calling sched_get_priority_min(SCHED_FIFO) and
sched_get_priority_max(SCHED_FIFO) respectively.


6.23 What is the mutex Protocol Attribute?

The Protocol Attribute provides another method by which the priority of
threads which gain ownership of a mutex may have their priority
automatically adjusted (see the section discussing the Ceiling Priority
Attribute for the another method). The following protocols are defined:

PTHREAD_PRIO_NONE

A thread which gains ownership of a mutex with a Protocol Attribute of
PTHREAD_PRIO_NONE, its priority and scheduling will not be affected.

PTHREAD_PRIO_INHERIT

A thread which has ownership of a mutex with a Protocol Attribute of
PTHREAD_PRIO_INHERIT will:

a) If the thread is not blocking other threads or is blocking threads of
lower priority due to its ownership of the mutex, the priority of the
thread will remain unchanged.

b) If the thread is blocking at least one thread of higher priority due
to its ownership of the mutex, the thread will have its priority raised
to the same level as the currently blocked thread with the highest
priority.

PTHREAD_PRIO_PROTECT

A thread which has ownership of one or more mutexes with a Protocol
Attribute of PTHREAD_PRIO_PROTECT, it will have its priority raised to
the highest Priority Ceiling specified by all the mutexes. As can be
seen, when this Protocol Attribute value is specified, the Priority
Ceiling attributes of the mutexes should also be specified for the
protocol to be effective. Also, if a thread with a higher priority than
the Priority Ceiling of the mutex ever attempts to gain a lock /
ownership of the mutex, the attempt will always fail!


Note:

"If a thread simultaneously owns several mutexes initialized with
different protocols, it shall execute at the highest of the priorities
that it would have obtained by each of these protocols." IEEE
Std.1003-2001


6.24 How can I determine if a system supports the mutex Protocol
Attribute?

If either the symbolic constant _POSIX_THREAD_PRIO_INHERIT or
_POSIX_THREAD_PRIO_PROTECT is defined in unistd.h to a value greater
than 0, the system provides support for the Protocol Attribute.

In particular:

a) If _POSIX_THREAD_PRIO_INHERIT is defined then the symbolics (and
associated protocols) PTHREAD_PRIO_NONE and PTHREAD_PRIO_INHERIT will be
available.

b) If _POSIX_THREAD_PRIO_PROTECTED is defined then the symbolics (and
associated protocols) PTHREAD_PRIO_NONE and PTHREAD_PRIO_PROTECT will be
available.

c) If _POSIX_THREAD_PRIO_INHERIT and _POSIX_THREAD_PRIO_PROTECTED are
defined then the symbolics (and associated protocols) PTHREAD_PRIO_NONE,
PTHREAD_PRIO_INHERIT and PTHREAD_PRIO_PROTECT will be available.



6.25 What functions get/set the mutex Protocol Attribute?

To query a mutex attribute variable for the current value of its
Protocol Attribute, use the function:

int pthread_mutexattr_getprotocol
(
   const pthread_mutexattr_t * attr,
   int * protocol
);


Where the parameter attr is a pointer to the location of the mutex
attribute object we want to query and protocol is a pointer to the
location where the obtained information will be placed.


To set the Protocol Attribute of a mutex attribute variable, the
following function is supplied:

int pthread_mutexattr_setprotocol
(
   pthread_mutexattr_t * attr,
   int protocol
);

Where the parameter attr is a pointer to the mutex attribute object you
wish to modify and the parameter protocol indicates the (new) desired
protocol and must be PTHREAD_PRIO_NONE, PTHREAD_PRIO_INHERIT or
PTHREAD_PRIO_PROTECT.

It is up to the programmer to determine if the desired protocol is
supported (see "How can I determine if a system supports the Protocol
Attribute?").


6.26 How do I query an existing mutex for its Protocol Attribute?

The Protocol Attribute is exposed to application code only via the mutex
attribute object used to initialize a mutex and cannot be obtained
directly from the mutex or modified once the mutex is created.


6.27 How do I modify the Protocol Attribute of an existing mutex?

The Protocol Attribute is exposed to application code only via the mutex
attribute object used to initialize a mutex and cannot be obtained
directly from the mutex or modified once the mutex is created.


6.28 What is deadlock?

Deadlock means that a thread is waiting for a resource, in this case a
mutex, which it can never obtain. Two common examples of deadlock are:

Example A:

A thread which obtains a lock on a mutex and then attempts to lock the
mutex a second time. Unless the mutex's Type Attribute has been set to
PTHREAD_MUTEX_RECURSIVE, the second lock attempt by the thread will
result in either an error being returned ( if of type
PTHREAD_MUTEX_ERRORCHECK ) or the thread in question blocking forever.


Example B:

Considering two threads labeled (A) and (B) and two mutexes labeled (0)
and (1) consider the scenario where thread (A) obtains a lock on mutex
(0). Thread (B) then obtains a lock on mutex (1). If thread (A) must
then obtain the lock on mutex (1) and thread (B) must gain a lock on
mutex (0), both threads will block forever waiting for the other one to
release the lock held on their initial mutex.

Unless you are writing recursive code, the deadlock scenario described
in Example B is the type you will probably most often see; though
probably in a much more complex form.


6.29 What is priority inversion (with mutexes)?

Priority inversion is the situation where a thread with a high(er)
priority is forced to suspend (wait) for acquisition of a lock on a
mutex while a low(er) priority thread holds the lock and continues to
execute.

The sections covering the mutex Priority Ceiling and Protocol attributes
details two methods by which priority inversion can be avoided by
automatically forcing a normalization of thread priorities as they gain
ownership of a particular mutex.



6.30 How many mutex objects can exist at the same time?

There is no explicit maximum count for the number of mutex objects which
may exist simultaneously within an application or on a system in
general. Considering this and the fact that mutexes are not required to
be created at the kernel level and kernel specific resources are not
consumed, most systems are normally limited only by the amount of memory
available.


6.31 How do I lock and unlock a mutex?

A thread can attempt to lock a mutex by calling the function:

int pthread_mutex_lock
(
   pthread_mutex_t * mutex
);

Where the parameter mutex points to a location which contains the mutex
the thread desires to lock. The function will not return until the
thread has obtained ownership of the indicated mutex. The mutex type
attribute, as outlined previously, will determine the conditions under
which error return values are generated.


A thread releases a lock (ownership) of a mutex by calling the function:

int pthread_mutex_unlock
(
   pthread_mutex_t * mutex
);

Where the parameter mutex points to a location which contains a mutex to
which the thread currently has ownership. The function returns a 0 on
success and and error of EPERM if the thread performing the unlock is
not the current owner of the mutex.

Of course, the thread which locks a mutex must also be the thread which
calls unlock. To allow otherwise would be to defeat the usefulness of
the mechanism.


6.32 Can I test a mutex to see if it is locked?

There is no function which allows for a direct test of a mutex to check
if the mutex is currently owned by a thread.

Instead the POSIX API provides the function:

The function is:

Code:

int pthread_mutex_trylock
(
   pthread_mutex_t * mutex
);


This function is identical in behavior to pthread_mutex_lock( ) except
when the mutex is currently owned by another thread. In such a
situation, this function will return an error number of EBUSY to
indicate that the calling thread could not gain immediate ownership.


6.33 Can I tell a thread to stop waiting to gain ownership of a mutex
once it starts waiting?

If a thread calls pthread_lock( ) and is queued because another thread
currently has ownership of the indicated mutex, there is literally no
way to portably tell the thread at some arbitrary point in time that it
should return from the function call without gaining ownership itself.

See the sections titled "Can I test a mutex to see if it is locked?" and
"Can I put a time limit on the amount of time a thread waits to obtain
ownership of a mutex?" for alternative solutions.


6.34 Can I put a time limit on the amount of time a thread waits to
obtain ownership of a mutex?

If the symbolic constant _POSIX_TIMEOUTS is defined in unistd.h to a
value greater than 0 (preferably 200112L) then the system provides the
following function:

Code:

int pthread_mutex_timedlock
(
   pthread_mutex_t * mutex,
   const struct timespec * abs_timeout
);

This function is identical in behavior to pthread_mutex_lock( ) except
that when the mutex is currently owned by another thread, the calling
thread will wait until the time specified by abs_timeout trying to
obtain the lock.  If the expiration time is reached without the mutex
ownership being granted the function will return the error ETIMEDOUT.

Note: How abs_timeout is interpreted depends on the resolution of the
clock used by the system. From experience and due to the widely varying
resolutions employed on available systems, I personally do not rely on
any granularity higher than second intervals.


6.35 A thread crashed that owned a mutex, how do I unlock it?

In pthreads, the mutex is permanently locked due to the requirement that
the thread which has ownership of a mutex MUST be the thread which
relinquishes (unlocks) it.

There is NO standard facility for the creation of a "master" thread
which has extra privileges and which would be capable of performing such
an operation.


Some systems do have custom commands and controls for this type of
situation. For example in Solaris the mutex has an extra attribute that
is set to handle this situation via:

pthread_mutexattr_setrobust_np( mutex, PTHREAD_MUTEX_STALLED_NP );

After this call, if the owner of the mutex dies then the mutex is
automatically unlocked and the next thread that tries to gain ownership
would receive a EOWNWERDEAD error. Of course you still have to contend
with the possibility that your data wrapped or manipulated by the mutex
is in an inconsistent state.


6.36 Is a read write lock a mutex?

A read write lock can be viewed as a special type of mutex with its own
locking, unlocking and manipulation functions.

Despite similarities to generic mutexes, it is not the same as a mutex
and as such, is covered in detail in its own section of the FAQ labeled
"What is a Read Write Lock".


6.37 Is a spin lock a mutex?

A spin lock can be viewed as a special type of mutex with its own
locking, unlocking and manipulation functions.

Despite similarities to generic mutexes, it is not the same as a mutex
and as such, is covered in detail in its own section of the FAQ labeled
"What is a Spin Lock".


6.38 Which thread gets mutex ownership if several are queued for it?

If several threads are blocked on a locked mutex and the thread holding
the mutex releases it, the the system will normally decide which thread
gets ownership next by:

* looking at the scheduling policies of the blocked threads and order
them according to an implementation specific preference. 

* ordering the threads with the prefered scheduling policy by priority.

* if there is more than one thread with the same priority then give
the lock to the one which either would get the next available time
slice on the CPU.

I say this is the normal method but of course it is not necessarily
the exact methodology an implementation might use for deciding on
mutex ownership.  Still, I believe it a fair outline of what you
could typically expect.



6.39 How do I emulate a recursive mutex?

On some systems you may run into the situation where the recursive mutex
type does not exist. In such a case there are two basic alternative
approaches which do a fair job of emulating the behavior of a recursive
mutex in user code.

If the system does NOT understand PTHREAD_MUTEX_RECURSIVE but does have
PTHREAD_MUTEX_ERRORCHECK then set your mutex to that type. The reason is
that the error checking mutex which will cause an EDEADLK errno to be
returned immediately on an attempted recursive lock instead of
deadlocking forever and to act in the normal manner for a non-recursive
lock attempt. Do this and also adding a static counter so you know when
you have backed off all the recursive calls and its time to unlock the
mutex you get a faked recursive mutex. On basic example of this is as
follows:

#define _POSIX_C_SOURCE 200112L
#define _XOPEN_SOURCE 600

int locking_count = 0;

pthread_mutex_t mutex;
pthread_mutexattr_t attr;

pthread_mutexattr_settype( & attr, PTHREAD_MUTEX_ERRORCHECK );
pthread_mutex_init( & mutex, & attr );

void some_func( )
{
  int result;

  if (0 != (result = pthread_mutex_lock( & mutex ) ) && EDEADLK != result)
  {
    /* Some other bizarro err other than recurse locking */
  }

  ++ locking_count;

  /** RECURSIVE CODE BODY **/

  -- locking_count;

  if ( 0 == locking_count )
  {
    pthread_mutex_unlock( & mutex );
  }
}


If by some chance the system doesn't supply either recursive or error
checking mutex types but DOES provide pthread_trylock then you could do
something along the lines of:

#define _POSIX_C_SOURCE 200112L
#define _XOPEN_SOURCE 600

pthread_t locking_id;
int locking_count = 0;

pthread_mutex_t mutex;
pthread_mutexattr_t attr;
pthread_mutexattr_settype( & attr, PTHREAD_MUTEX_NORMAL );
pthread_mutex_init( & mutex, & attr );

void some_func( )
{
  /** 
   Following should probably check that errno is EBUSY also
  **/

  if ( 0 != pthread_mutex_trylock( & mutex ) )
  {
    if ( 0 == pthread_equal( pthread_self( ), locking_id ) )
    {
      pthread_lock( & mutex );
    }
  }

  locking_id = pthread_self( );

  ++ locking_count;

  /** RECURSIVE CODE BODY **/

  -- locking_count;

  if ( 0 == locking_count )
  {
    locking_id = some unused thread id;

    pthread_mutex_unlock( & mutex );
  }
}


Of course these are examples and the code could be written many
different ways in either of the described situations.



6.40 What is thread starvation?

The term thread starvation or simply "starvation" is the term used to
describe the condition where a thread is, for any reason, denied access
to CPU cycles or code segments that it requires to complete a task in a
timely manner.

There are two common causes of CPU cycle starvation. They are:

* The threads have been assigned different scheduling attributes and the
implementation is giving preference to threads using one of the
scheduling policies.

* The threads have been given priorities which vary greatly resulting in
the lower priority thread(s) never being scheduled.

The solution to the first problem is to try and not mix and match the
various scheduling schemes within a single program unless you explicitly
know their interaction on the target system.

To solve the second issue, it is ( normally ) best to keep the majority
of threads at a single identical priority level and only manipulate the
priority of "special" threads. For example, if you had a synchronous
signal handler in your program you may want raise it's priority to make
certain that it processed signals quickly but would leave all other
threads at their default priority level.


Starvation caused by being blocked from a specific code segment can come
from:

* A coding flaw where two or more threads have a co-dependency on mutual
exclusion locks already held by each other. This is actually a special
case called "deadlock" and is described in the like named topic in this
section.

* One or more threads in a group of threads trying to gain an exclusion
lock are being given preference either due to an implementation
preference for the thread(s) scheduling policy or because they have a
higher priority setting.

The first issue can be solved by careful planning and coding while for
the second I would make the same suggestions as were discussed under CPU
cycle starvation.

Finally, starvation of both types can be caused by any number of pthread
implementation flaws on the target system. In fact the previous version
of threads on a certain well known OS could exhibit, in condensed form,
the following behavior where two or more threads were in a loop
competing for the same exclusive lock:

* Thread A tries to gain exclusive lock
* Thread A gains the exclusive lock
* Thread B tries to gain exclusive lock
* Thread B blocks
* Thread A begins processing in critical section
* System scheduler determines Thread A timeslice is expired
* System scheduler determines Thread B should get next timeslice
* System scheduler determines Thread B cannot execute since blocked
* System scheduler gives new timeslice to Thread A
* Thread A finishes processing in critical section
* Thread A releases the exclusive lock
* Thread A has remaining cycles left in current timeslice
* Thread A loops
* Thread A gains the exclusive lock.
* Repeat FOREVER

If you suspect something like this is happening about the only
suggestion I can give is to make a call to the function sched_yield( )
immediately after any thread releases the exclusive lock. Performing
this call will eat up a few extra CPU cycles but with any luck it will
also give enough of a hint to the scheduler so that a timeslice is
allocated to one of the blocked threads.


6.41 Can I use "volatile" instead of a mutex?

It is a common misconception that marking a variable volatile will
provide synchronization for the variable if it is only having simple
operations being performed on it..  

Take the follow ( incorrect ) code example:

<pre class="codeBlock">
volatile int index;

thread_func_A( )
{
  ++ index;
}

thread_func_B( )
{
  ++ index;
}
</pre>

Here we see a basic integer variable named index being incremented by
two individual threads.  The assumption goes that the ++ index command
is translated into a single inc machine language command.  Since it is
a single machine command it must be safe.

Lets look at it closer though so we can see what is really happening at
the lower level.  Even for something as basic as int, especially when
it is declared volatile, the processor still must read the value, then
actually increment it and finally store it back into memory.  During
this process there is nothing to prevent a secondary processor from
performing the same action and still causing a collision.  

The moral of the story is do not use the volatile keyword in the hope
that it will provide synchronization.  That is not the purpose of the
keyword.  If you require a variable to be synchronized then use a
proper mutual exclusion lock to protect it.



6.42 Are mutexes correct if I usually just read data in the critical
  section?

Lets first point out that if you ONLY read the data and NEVER perform
a write operation that you do not need a mutex.  On the 
other hand, if you usually perform read operations and only 
infrequently do writes then full mutex will certainly work but may
not be a efficient as the alternative of using a read-write lock.

For more information on read-write locks see section 10 of this document.



6.43 A simple example of using a mutex

The following is a simple example of a program which uses mutexes. It
doesn't accomplish anything except to print out the an identifier for
the threads as they obtain / lock the mutex.

It does NOT contain any error checking simply to reduce the length of
the code.

#define _POSIX_C_SOURCE 200112L

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <string.h>
#include <sys/select.h>

#define NUM_THREADS 5

pthread_mutex_t mutex;


void * simple_thread ( void * arg )
{
   int i;

   pthread_t thread_id = pthread_self( );

   struct timeval timeout;

   for ( i = 0; i < 5; ++ i )
   {
     pthread_mutex_lock( & mutex );

     printf( "Thread %li has lock on critical section\n",
             ( long int ) & thread_id
     );

     pthread_mutex_unlock( & mutex );

     /* Force the thread to pause for a half second */

     timeout.tv_sec = 0;
     timeout.tv_usec = 500;

     select( 0, NULL, NULL, NULL, &timeout );
   }

   return NULL;
}

int main ( int argc, char * argv[ ] )
{
   int i;

   pthread_t thread_id[ NUM_THREADS ];

   pthread_mutex_init( & mutex, NULL );

   for ( i = 0; i < NUM_THREADS; ++ i )
   {
     pthread_create ( & thread_id[ i ], NULL, simple_thread, NULL );
   }

   for ( i = 0; i < NUM_THREADS; ++ i )
   {
     pthread_join ( thread_id[ i ], NULL );
   }

   return 0;
}



7. POSIX CONDITION VARIABLES

Condition variables allow you to suspend the execution of a thread until
some pre-determined condition occurs. Multiple threads may block on a
single condition variable at the same time AND the suspended threads may
be signaled to wake up either individually or as a group.

One of the most common uses of condition variables is in producer -
consumer type setups where the consumers are suspended until the
producer has data available for one of them. Of course this is just one
use of the many possible.

A last note on condition variables and thier use. They do not provide
any locking or other synchronization than what has already been
described. Because of this, a condition variable is always used in
conjunction with a mutex. You can see examples of this in the sections
on pthread_cond_wait( ) and related functions.


7.1 How do I determine if my implementation supports condition
variables?

If your system provides an implementation of POSIX threading then it
will also provide definitions for basic condition variable creation and
manipulation.

If a system defines the symbolic constant _POSIX_THREADS in the file
unistd.h to a value greater than 0, it provides an implementation of
POSIX threading.



7.2 How do I create a condition variable?


Condition variables are of type pthread_cond_t so the first step of
creating an object of this type for use is to include a declaration in
your application source code as follows:

pthread_cond_t [CONDITION_VARIABLE_NAME];


Even though the condition variable has been declared and will be
instantiated on execution of the given code, it is still not ready for
use. Prior to the first use of the condition variable it must be
initialized by calling the function:


int pthread_cond_init
(
   pthread_cond_t * restrict cond,
   const pthread_condattr_t * restrict attr
);


Where the cond parameter specifies the location of the condition
variable we wish to initialize and the attr parameter specifies the
desired condition variable attributes. If the attr parameter is NULL,
the default attributes are used by the system. See the next few sections
for details about the pthread_condattr_t type, its creation and the
manipulation of condition variable attributes.

On success the function will return a value of zero and the condition
variable will be ready for use. The call may fail with either EAGAIN or
ENOMEM if there are not enough resources or memory respectively.

You may also see an error of EBUSY if you try to re-init a condition
variable that was previously initialized but hadn't been destroyed. I
use the word "may" because that is what the spec says and it is
implementation independent.


It is also possible to declare and initialize a static condition
variable using the following:

static pthread_cond_t [CONDITION_VARIABLE_NAME] =
PTHREAD_COND_INITIALIZER;

The use of PTHREAD_COND_INITIALIZER is the same as calling
pthread_cond_init( ) with an attribute parameter of null ( default
values ). You should be aware of the fact that if you use this
initializer and an error occurs, you will never be notified of the
error.


7.3 How do I destroy a condition variable?

As with most things in the pthreads section, once you are finished using
a condition variable, you should destroy it. Failure to do so will
result in the resources consumed by the condition variable to never be
released back to the system with the end result being a memory leak.

The function which handles destroying a condition variable is:

int pthread_cond_destroy
(
   pthread_cond_t * cond
);

Where the parameter cond is the condition variable you want to release.

This call will return an EINVAL if the cond parameter is invalid and can
also yield a EBUSY if you are attempting to destroy a condition variable
which is still referenced e.g. by a thread still suspended on a
pthread_cond_wait( ) call.


7.4 What is a condition variable attribute?

A couple of the default behaviors of a condition variable can be
manipulated through the use of a condition variable attribute. You
probably won't use them very much unless you need to change the default
behavior of the condition variable.

The condition variable attribute data type is defined in the file
pthread.h and specified to be pthread_condattr_t.


7.5 How do I create a condition variable attribute?

To create a condition variable ready for use with default values simply
declare it using something like:

pthread_condattr_t [COND_VAR_NAME];

Then you must call the function:

int pthread_condattr_init
(
   pthread_condattr_t * attr
);

Where the parameter attr is the uninitialized condition variable.

This function will return a zero if it was successful and a ENOMEM if
the system has insufficient memory to initialize the attribute and it's
underlying resources.

See the next section on how to destroy a condition variable attribute
and resources which are allocated during initialization. Remember that
failure to do this can result in a hard to track down slow memory leak.


7.6 How do I destroy a condition variable attribute?

After you have finished using a condition variable attribute you should
destroy it so that any system resources allocated to during
initialization are given back to the system for re-use.

Destruction of a condition variable attribute is done via the function:

int pthread_condattr_destroy
(
   pthread_condattr_t * attr
);

Where the parameter attr is the previously initialized condition
variable that we want to destroy.

A zero is returned if the call is successfull and a EINVAL if the attr
parameter is invalid in any way.


7.7 What are the available condition variable attributes?

There are two attributes defined by the current standard and they are
the clock attribute and the process-shared attribute. Most systems
should support the process-shared attribute but only those providing
advanced real time support will have the clock attribute.

Finally, a system may have implementation specific attributes but none
of those are mentioned in this document.


7.8 What is the condition variable Clock Attribute?

TO DO.


7.9 What is the condition variable Process-Shared Attribute?

The default for this attribute is PTHREAD_PROCESS_PRIVATE and is
specified to indicate that only those threads within the process which
created the condition variable are allowed to manipulate it.

If the attribute is set to PTHREAD_PROCESS_SHARED, then the condition
variable may have operations performed on it by any thread that has
access to the memory where it is allocated, even if the other threads
were created and contained by a process other than the one which created
the condition variable.

It a condition variable with its shared attribute set to
PTHREAD_PROCESS_PRIVATE is accessed from another process the result of
any such operation is undefined.

Since the use of a condition variable requires the presence of a mutex,
both the condition variable and mutex used should have the same type
share attribute. See the appropriate section on mutex attributes for
more information.



7.10 How can I determine if a system supports the Process-Shared
Attribute?

You can check unistd.h and if the symbolic constant
_POSIX_THREAD_PROCESS_SHARED is defined to a value greater than zero,
the system provides support of the Process-Shared Attribute.


7.11 What functions get/set the Process-Shared Attribute?

To query a condition variable attribute for the current state of the
Process-Shared Attribute, use the function:

int pthread_condattr_getpshared
(
   const pthread_condattr_t * restrict attr,
   int * restrict pshared
);

Where the parameter attr is a pointer to the location of the condition
variable attribute object we want to query and pshared is a pointer to
the location where either the value PTHREAD_PROCESS_PRIVATE or
PTHREAD_PROCESS_SHARED will be placed.


To set the Process-Shared Attribute of a condition variable attribute,
the following function can be used:

int pthread_condattr_setpshared
(
   pthread_condattr_t * attr,
   int pshared
);


Where the parameter attr is a pointer to the condition variable
attribute object you wish to modify and the parameter pshared indicates
the desired sharing behavior.


7.12 How do I query an existing condition variable attribute for its
Process-Shared Attribute?

The Process-Shared Attribute is exposed to application code only via the
condition variable attribute object used to initialize the condition
variable. As such, it cannot be obtained directly from the condition
variable or modified once the condition variable is initialized.


7.13 How do I use a condition variable - using pthread_cond_wait( )?

The first requirement for using a condition variable is to create a
mutex object to use along with it. The initializatiion of a mutex is
covered in the appropriate section(s) of this document.

Once you have initialize a condition variable and mutex then you MUST
first obtain a lock on the mutex. Once you have the mutex lock then the
thread which holds the mutex lock can use the pthread_cond_wait( )
function to cause the calling thread to suspend on the condition
variable until it is "signaled". By "signaled" we do not mean signals in
the normal sense but that a call to either pthread_cond_signal( ) or
pthread_cond_broadcast( ) is performed. Those functions are covered in
detail further on.

The function prototype for pthread_cond_wait is:

int pthread_cond_wait
(
   pthread_cond_t * restrict cond,
   pthread_mutex_t *restrict mutex
);

Where the parameter cond is the condition variable and mutex is the
mutex previously locked by the calling thread.

When this function executes, the calling thread is forced ( internally )
to release the exclusive lock on the mutex. After the mutex is released,
the thread is suspended until signaled. If the thread receives a signal
and is resumed it will immediately attempt to re-establish the exclusive
lock on the mutex. This function will not return until the calling
thread the thread has gained that lock.

A couple of warnings:

* You should NEVER have multiple calls to pthread_cond_wait( ) or it's
timed version with the same condition variable and different mutexes.

* You should avoid mixing and matching condition variables and mutexes
which have different Process-Shared attributes e.g. one process private
and the other process shared.

* You should always check the condition after a thread has returned from
a wait. The reason for this is that systems can sometimes cause multiple
threads to be unblocked even when the pthread_cond_signal function (
unblock one thread ) is used.

Since a call to pthread_cond_wait( ) suspends the calling thread until
it receives a signal the potential exists for the caller to be suspended
forever. If you want to place an upper limit on the amount of time the
thread will suspend you should use pthread_cond_timewait( ) described
below.




7.14 How do I do a timed wait on a condition variable?

If you want to make certain that there is a maximum time limit that the
calling thread is suspended on a condition variable you should use
pthread_cond_timedwait( ) instead of the pthread_cond_wait( ) call.

The pthread_cond_timedwait( ) function is defined as:

int pthread_cond_timedwait
(
   pthread_cond_t * restrict cond,
   pthread_mutex_t * restrict mutex,
   const struct timespec * restrict abstime
);

The first two parameters are the same as those used by
pthread_cond_wait( ) while the new parameter abstime describes the
desired ABSOLUTE time when the thread should unblock and attempt to
regain a lock on the mutex and return from the function. If the absolute
time has already passed

The timespec structure is defined in time.h ( which should be included )
as requiring the following fields:

time_t tv_sec Seconds
long tv_nsec Nanoseconds

To determine the current system time ( so that you can calculate a
proper exit time for the timed wait, use the clock_gettime( ) function
in time.h and specify a clock_id of CLOCK_REALTIME e.g.:

clock_gettime( CLOCK_REALTIME, struct timespec * tp );

If a call to the function returns because of a call to
pthread_cond_signal or pthread_cond_broadcast then it will return a
zero. On the other hand, if it is returning because the time specified
by asbtime has already occurred, then the return value will be
ETIMEDOUT.

Of course you can also get an EINVAL error if any of the parameters are
invalid including the situation where the calling thread does not
currently own the lock on the specified mutex object.

A code segment with no error checking showing a timed wait:

/* This uses real time definition so add -lrt to gcc line */

#define _POSIX_C_SOURCE 200112L
#define NUM_LOOPS 5

#include <pthread.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <errno.h>

int main ( int argc, char ** argv )
{

   pthread_mutex_t mutex;
   pthread_cond_t condvar;
 

   struct timespec abstime;

   int result;

   int predicate;


   ( void ) pthread_mutex_init( & mutex, NULL );
   ( void ) pthread_cond_init( & condvar, NULL );

   ( void ) pthread_mutex_lock( & mutex );

   ( void ) clock_gettime( CLOCK_REALTIME, & abstime );

   /* One minute from the current system time */

   abstime.tv_sec += 60;

   predicate = 0;
   result = 0;

   /* The predicate is an indicator of the desired condition */
   /* that would be true if a valid call to signal was recvd */

   while ( ! predicate && ETIMEDOUT != result )
   {
     /* The wait is wrapped in a loop to catch sporadic and */
     /* erroneous wake ups that can occur on some machines */

     result = pthread_cond_timedwait( & condvar, & mutex, & abstime );
   }
  

   if ( 0 == result )
   {
     /* Take some action here because predicate indicates */
     /* conditions met and we did not time out / error out */
   }

   ( void ) pthread_mutex_unlock( & mutex );

   return 0;
}


7.15 How do I do a timed wait on a condition variable using a relative
time?

There is no function defined for waiting on condition variables that
take a relative time for the maximum wait period.

It is conceivable that you could implement your own function that
performed such a task but it would require extra work. Remember also
that using a relative time would mean that any time the condition
variable was unblocked sporadically / accidentally by the system you
would have to detect that fact AND recalculate the relative time.

Using an absolute time as specified with pthread_cond_timedwait( )
avoids this required recalculation and other potential issues.


7.16 Why didn't the timed wait return after I modified the system clock?

The specification states that if the clock is advanced and the absolute
time specified would have expired during the time that was skipped then
it is "expected" for the wait to be processed as though the time had
actually expired.

I think the keyword there is "expected". While many if not most
implementations actually provided the expected behavior there are some
which, when you advance the system clock past a specified absolute timed
wait will then ignore the time and just sit waiting for the a signal or
broadcast call.

The specification also says nothing about moving the system clock
backwards. Again, it has been noted that some ( older ) pthread
implementations will exhibit the same behavior as when skipping forward
past the specified time. That is, they will then ignore the time given
and rely only on a call to pthread_cond_signal or broadcast to be
unblocked.

So it seems the moral of the story is, if you have a pthread program
running which performs calls to pthread_cond_timedwait( ) its probably
best NOT to be moving the system clock in either direction. At least not
if you desire the expected behavior from the call.



7.17 How do I wake one thread suspended on a condition variable?

If one or more threads have blocked on a mutex and condition variable
combination either through calling pthread_cond_wait( ) or
pthread_cond_timedwait( ), then we need a way to notify one or more of
them to unblock and return from the call. To wake only one of the
blocking threads then we can use the function pthread_cond_signal( )
that was mentioned earlier.

This function is defined in pthread.h as:

int pthread_cond_signal
(
   pthread_cond_t * cond
);

Where the parameter cond is the condition variable used by the thread(s)
currently suspended in pthread_cond_wait( ) or pthread_cond_timedwait( )
functions.

When this call is made, one of two things will occur:

* If no threads are currently suspended on the condition variable then
NO action is taken.

* If one or more threads are currently suspended on the condition
variable then one will be chosen based on the current scheduling policy
of the system to be resumed and continue execution.

There may be circumstance where you want to wake ALL of the threads
suspended on a condition variable. The first option would be to do N
calls to pthread_cond_signal( ) where N is the known maximum number of
threads that could potentially be blocking. That wouldn't be very
efficient so there is a "broadcast" alternative supplied by POSIX and
described in the next section.


7.18 How do I wake ALL threads suspended on a condition variable?


If you have more than one thread blocked on a mutex and condition
variable combination either through calling pthread_cond_wait( ) or
pthread_cond_timedwait( ) and you want to wake all of them up then use
pthread_cond_broadcast( ).

The broadcast function is defined as:

int pthread_cond_broadcast
(
   pthread_cond_t * cond
);

Where the parameter cond is the condition variable used by the threads
currently suspended in pthread_cond_wait( ) or pthread_cond_timedwait( )
functions.

When this call is made, one of two things will occur:

* If no threads are currently suspended on the condition variable then
NO action is taken.

* If one or more threads are currently suspended on the condition
variable then ALL the threads will be resumed and will ALL attempt to
gain a lock on the associated mutex.

You should also read the next section titled "What is the thundering
herd problem?".


7.19 What is the thundering herd problem?

In the situation where you have threads suspended on a signal variable
and you use pthreda_cond_signal( ), only one of the threads is resumed.
This means that there is no contention for CPU time or from attempts to
gain an exclusive lock on the associated mutex.

On the other hand, if you have multiple threads suspended and call
pthread_cond_broadcast( ) then ALL the threads are resumed. When this
occurs its often called a "thundering herd".

The reason for name "thundering herd" is that all the threads are
resumed and immediately begin contending / rushing for time slice on the
CPU to continue execution. Not only that, they also start simultaneous
to contend to get the lock on the single associated mutex. Its really
much like a herd of animals rushing blindly at those resources and often
causes huge spikes in CPU consumption which is something to be avoided.

Having said that, my personal suggestion is to use pthread_cond_signal(
) where ever it is possible. Only use the broadcast version if you know
for a fact that you require ALL of the thread to be unblocked e.g.
during a server shutdown.


7.20 A small example of using a condition variable


/** NO ERROR CHECKING PERFORMED **/

#define _POSIX_C_SOURCE 200112L
#define NUM_LOOPS 5

#include <pthread.h>
#include <stdio.h>
#include <time.h>
#include <sys/select.h>


pthread_mutex_t mutex;
pthread_cond_t condvar;

int predicate;


void * start( void * arg )
{
   int count = 0;

   int result;

   while ( NUM_LOOPS > count )
   {
     result = 0;

     ( void ) pthread_mutex_lock( & mutex );

     while ( 0 == predicate && 0 == result )
     {
       result = pthread_cond_wait( & condvar, & mutex );
     }

     ( void ) printf( "Thread unblocked from pthread_cond_wait( ).\n" );
  

     -- predicate;

     ( void ) pthread_mutex_unlock( & mutex );

     ++ count;
   }

   return NULL;
}


int main ( int argc, char ** argv )
{
   pthread_t thread;

   int count;

   struct timeval timeout;

   ( void ) pthread_mutex_init( & mutex, NULL );
   ( void ) pthread_cond_init( & condvar, NULL );

   predicate = 0;

   ( void ) pthread_create( & thread, NULL, & start, NULL );

   count = 0;

   while ( NUM_LOOPS > count )
   {
     /* Set the predicate and generate a pthread_cond_signal( ) */
     /* call approximately every 3 seconds to unblock the child */

     timeout.tv_sec = 3;
     timeout.tv_usec = 0;

     ( void ) select( 0, NULL, NULL, NULL, & timeout );

     ( void ) pthread_mutex_lock( & mutex );

     ++ predicate;

     ( void ) pthread_mutex_unlock( & mutex );

     ( void ) pthread_cond_signal( & condvar );

     ++ count;
   }

   return 0;

}





8. PTHREADS & SIGNALS

The behavior of signals in a multi-threaded program is quite different
from the behavior in a non-threaded program. Here is a list of the basic
things you can expect from mixing threads and signals:

* if a process receives a signal then one and only one indeterminate
thread in the process which is not blocking the signal will receive the
signal.

* if a process receives a signal and more than one thread is waiting on
that signal, then one and only one indeterminate thread out of that
waiting group will receive the signal.

* if any signal which would normally cause the termination of a process
is sent to a thread it will result in the parent process and all threads
being terminated.

* on creation a thread inherits the signal mask of the thread that
created it.


Some of the functions used to manipulate and receive signals are also
different with the primary ones discussed being:

sigemptyset( )
sigfillset( )
sigaddset( )
sigdelset( )


pthread_sigmask( )

sigwait( )
sigwaitinfo( )
sigtimedwait( )

pthread_kill( )


One last note on this section. It comes nowhere near to providing an in
depth discussion of signal handling with threads. For now it simply
explains some of the more basic aspects performing synchronous signal
handling. With any luck more information will be added as time goes on.



8.1 What is a signal set and how do I create one?

A signal set is nothing more than a variable of type sigset_t as defined
in the signal.h header file. In pthreads and POSIX in general it is used
to set and retrieve the signals blocked / unblocked by a process or
thread and to indicate interest in specific signals to which the
sig...wait( ) functions will respond.

Prior to first use a signal set should be initialized using one of two
functions. The first function that we will look at is sigemptyset. A
call to this function will initialize the given signal set so that all
signals are EXCLUDED from the set.

The sigemptyset( ) function has the following prototype:

int sigemptyset
(
   sigset_t * set
);

The second function that we will look at is sigfillset. As you can
imagine this does exactly the opposite of sigempty set. That is to say,
a call to this function will initialize the given signal set so that all
signals are INCLUDED in the set.

int sigfillset
(
   sigset_t * set
);

One thing about sigemptyset( ) and sigfillset( ) is that while both
return integer values, neither have any possible error conditions
defined so the return is safely ignored. Once you have done the
initialization of the signal set, it would normally be time to do some
fine grained adjustments and add or remove specific signals from the
set.

First we will look at adding a single signal to a set. This operation is
facilitated using the sigaddset( ) function shown below:

int sigaddset
(
   sigset_t * set,
   int signo
);

Where signo is the signal to add to the signal set "set".

This can throw an error and return a value of -1 placing EINVAL in
errno. This will only occur if the value of signo is invalid or
unsupported on the target system.

To delete a single signal from a set the sigdelset( ) function is
provided. The prototype is:

int sigdelset
(
   sigset_t * set,
   int signo
);

Where signo is the signal to add to the signal set "set".

This can throw an error just like sigaddset, in particular giving a -1
and EINVAL in the errno variable. As with sigaddset this will only occur
if the value of signo is invalid or unsupported on the target system.

A quick code fragment showing the use of some of these functions and
creating a signal set that only contained SIGINT ( control-c interrupt )
would be:


#include <signal.h>

...

/* The signal set we will manipulate */

( void ) sigset_t signal_mask;

/* Init the signal set so that it is empty */

( void ) sigemptyset( & signal_mask );

/* Now add SIGINT to the signal set */

( void ) sigaddset( & signal_mask, SIGINT );



8.2 How can I tell if a signal is in a specific signal set?

If you have a signal set and want to determine what signals are ( or are
not ) contained in it the function to use is defined in signal.h as
follows:

int sigismember
(
   const sigset_t * set,
   int signo
);

Where signo is the signal you want to check for existence in the signal
set parameter "set". The function will return 0 if signo DOES NOT exist
in the set and a 1 if it DOES exist.

The only possible error is EINVAL which will be indicated by a return
value of -1 and passed via errno if the value of signo is invalid or
unsupported on the target system.



8.3 How do I get the signal mask of a thread?

The same function is used to both get and set the signal mask of a
thread. It is defined in pthread.h as:

int pthread_sigmask
(
   int how,
   const sigset_t * restrict set,
   sigset_t * restrict oset
);

To perform retrieval the how parameter should be SIG_BLOCK ( or actually
any of the valid constants since retrieval ops should ignore this value
). Additionally the set parameter should be null and the oset parameter
should be a previously created and cleared signal set. On return, the
oset parameter will contain the thread's signal mask.

For example:

#include <pthread.h>
#include <signal.h>

...

sigset_t my_set;

sigemptyset( & my_set );

( void ) pthread_sigmask( SIG_BLOCK, NULL, & my_set );


The pthread_sigmask should not return an error during a get operation
except in the situation where you put an invalid value in the "how"
parameter. Such a case is an obvious error and will cause the function
to return EINVAL.



8.4 How do I set the signal mask of a thread?

As mentioned in the section on getting signal masks, the same function
is used to both get and set the signal mask of a thread. It is defined
in pthread.h as:

int pthread_sigmask
(
   int how,
   const sigset_t * restrict set,
   sigset_t * restrict oset
);

Where the allowed parameter values are:

how: Indicates how the thread's signal mask will be manipulated and can
be one of the following three values.

   SIG_BLOCK: The new signal mask for the thread will consist of the
union of the current set and the signal set pointed to by set.

   SIG_SETMASK: The new signal mask for the thread will consist of the
signal set pointed to by set.

   SIG_UNBLOCK: The new signal mask will consist of the intersection of
the current set and the complement of the signal set pointed to by set.

set: The new signal set to apply to the thread in accordance to the
rules specified by the given "how" value. If this value is null then
oset must be non-null AND a get operation will be performed.

oset: The signal set where the thread's prior signal mask will be
placed. If a copy of the prior mask is not desired the value of this
parameter can be null.

The only error condition defined for this function is the use of an
unknown value for the "how" parameter. In such a case, the error value
of EINVAL will be returned.

For example:

#define _POSIX_C_SOURCE 200112L

#include <pthread.h>
#include <signal.h>
#include <stdio.h>

int main ( int argc, char ** argv )
{
   int sig_num;

   sigset_t new_set;
   sigset_t old_set;

   sigemptyset( & new_set );
   sigemptyset( & old_set );

   ( void ) sigaddset( & new_set, SIGINT );

   /* Set the signal mask of the thread */

   ( void ) pthread_sigmask( SIG_BLOCK, & new_set, & old_set );

   /* Get the modified signal of the thread */

   ( void ) pthread_sigmask( SIG_BLOCK, NULL, & new_set );

   for ( sig_num = 0; sig_num < SIGUSR2; ++ sig_num )
   {
     if ( 0 != sigismember( & old_set, sig_num ) )
     {
       printf( "Old set blocking signal: %i\n", sig_num );
     }
     else
     {
       printf( "Old set not blocking signal: %i\n", sig_num );
     }

     if ( 0 != sigismember( & new_set, sig_num ) )
     {
       printf( "New set blocking signal: %i\n", sig_num );
     }
     else
     {
       printf( "New set not blocking signal: %i\n", sig_num );
     }
   }

   return 0;
}



8.5 Can I use sigprocmask( ) instead of pthread_sigmask( )?

You can definitely use sigprocmask( ) instead of pthread_sigmask( ) if
your program is going to run as a single threaded process. If you try to
use it in a program that is running as multi-threaded process you WILL
get unexpected and undesirable results.

On the other hand pthread_sigmask( ) will work just fine with both
single and multi-threaded processes. Since this is the case there really
aren't many circumstances not to use pthread_sigmask( ).



8.6 What is the signal mask for a newly created thread?

Any thread that is created inherits the signal mask from it's parent. If
a signal mask different from the parent is required then modification of
the mask must be performed by the thread after it's creation.



8.7 Using sigwait( ) to receive and process a signal

The sigwait function is the easy way to "catch" signals in a synchronous
manner. In simplest terms by synchronous we mean that the thread calling
the function will suspended until at least one of the specified
signal(s) received.

The function prototype is:

int sigwait
(
   const sigset_t * restrict set,
   int * restrict sig
);

Where the parameters are defined as:

set: A signal set containing the signal(s) the wait function is willing
to receive.

sig: The signal, which will be one of those specified in "set", that was
received and caused the function to return.

The only possible error is if the set param contains an invalid or
unsupported signal in which case the function will return a -1 and
EINVAL in errno.

Some comments and common errors concerning sigwait( ) are:

* If you call sigwait( ) to wait on a particular signal or set of
signals then that signal or signals should under most circumstances also
be blocked in ALL threads including the thread which called sigwait( ).

* If you call sigwait( ) to wait on a particular signal or set of
signals and that signal or signal is not blocked in one or more other
threads it is indeterminate whether the waiting thread will return from
sigwait( ) or if one of the threads with the signal(s) unblocked will
receive that signal in an asynchronous manner.

* If you call sigwait( ) to wait on a particular signal or set of
signals in more than one thread then ONLY ONE waiting thread will return
from sigwait( ).

* If you call sigwait( ) to wait on a particular signal or set of
signals in more than one thread then it is indeterminate WHICH of the
waiting threads will actually return from sigwait( ).

* If you call sigwait( ) you will only be provided with the signal which
was received and no other information. See sigwaitinfo( ) and
sigtimedwait( ) for information on how to retrieve additional
information on the signal.

* If you call sigwait( ) and none of the signals indicated in set are
ever received the calling thread will block forever unless the thread
catches another unblocked, caught signal. In such a case the function
will return a -1 and EINTR in the errno variable. See sigtimedwait( )
for information on doing a timed wait.


8.8 Using sigwaitinfo( ) to receive and process a signal

In many cases you require more information than what is provided by a
simple sigwait( ) call as described previously. In such circumstance you
can utilize the sigwaitinfo( ) call and have a siginfo_t structure
filled with extended data.

The sigwaitinfo prototype is:

int sigwaitinfo
(
   const sigset_t * restrict set,
   siginfo_t * restrict info
);

All behavior of this function is the same as a sigwait( ) except for the
fact that a siginfo_t structure is populated instead of a simple integer
indicating the received signal number.

While a full description of all the siginfo_t members is beyond the
scope of this document the basic definition taken directly from the
POSIX standard is:

int si_signo Signal number.
int si_errno If non-zero, an errno value associated
                         with this signal, as defined in <errno.h>.
int si_code Signal code.
pid_t si_pid Sending process ID.
uid_t si_uid Real user ID of sending process.
void * si_addr Address of faulting instruction.
int si_status Exit value or signal.
long si_band Band event for SIGPOLL.
union sigval si_value Signal value.



8.9 Using sigtimedwait( ) to receive and process a signal

Since both sigwait( ) and sigwaitinfo( ) both suspend the calling thread
until one of the specified signals is received or they are interrupted
by a unblocked and caught signal they can be hard to exit from at times.
To facilitate an easy and timed exit while waiting for a signal the
POSIX standard adds the function sigtimedwait( ) to it's definitions.

The prototype of sigtimedwait( ) is:

int sigtimedwait
(
   const sigset_t * restrict set,
   siginfo_t * restrict info,
   const struct timespec * restrict timeout
);

Where the set and info parameters are identical to those described under
the sigwaitinfo( ) section and:

timeout: A structure of type timespec which specifies the desired
maximum period to wait for a signal in the specified set to be received.

The timespec structure is defined in time.h ( which should be included )
as requiring the following fields:

time_t tv_sec Seconds
long tv_nsec Nanoseconds

A call to this function acts the same as a sigwaitinfo( ) call but has
an extra field which specifies the desired maximum amount of time the
thread should be suspended while waiting to receive one of the desired
signals. If this timer expires prior to the function receiving a signal
it will return a -1 and EAGAIN in the errno.

The function can also return a -1 and EINTR errno if the call is
interrupted by an unblocked and caught signal. This behavior is
identical to both the sigwait( ) and sigwaitinfo( ) functions.

Last but not least, if the tv_nsec value of the timeout structure is
less than zero or greater than 999 million the function will return
immediately with a -1 and an EINVAL errno.


8.10 How can I send a signal to a specific thread?

The function pthread_kill( ) in signal.h allows you to send a signal to
a specific thread in your process.

The function is defined as:

int pthread_kill
(
   pthread_t thread,
   int sig
);

Where the parameters are:

thread: The thread which will receive the signal.

sig: The signal to send to the target thread.

The function will return a zero on success, a ESRCH if the given thread
is not valid and a EINVAL is the signal specified is invalid or
unsupported.

If you are familiar with the kill( ) function then you are probably also
familiar with using it to send a signal value of 0 to a process so that
you can detect if the process is still alive. In the same way,
pthread_kill( ) can be used with a signal value of 0 to detect if the
specified thread is still alive.


8.11 What is the "recommended" setup for signal handling with threads?

This is only a recommendation and my personal recommendation at that. It
is meant to simplify signal handling by forcing the entire program to
operate ONLY in a synchronous manner and avoid many of the pitfalls
associated with asynchronous signal handling with pthreads.

* In your parent thread set the signal mask to block ALL signals
including those you want to monitor.

* In your child threads leave the default signal mask which would be the
same as the parent with ALL signals blocked.

* Create a single special child thread who has the sole task of calling
sigwait( ) or one of the advanced wait functions and receives ALL
signals.

* Create an "Observer / Observable" or similar type pattern relationship
between the regular child threads and the special child processing
signals.

Again, this ONLY applies to a program using synchronous signal handling.
If you are doing real time asynchronous signal handling then this setup
would not be appropriate or workable.


8.12 Why is the wrong thread receiving a signal?

Some other possibilities and things to check:

* the missing signal(s) is unblocked in more than one thread.

* the missing signal(s) is being waited on by more than one thread using
sigwait( ) calls.

* the code is has a combination of the first two items e.g. there exists
one thread with the signal unblocked and another thread waiting on it
with sigwait.

* the code is mixing non-threaded and pthread signal calls.

There is the possibility that you are using an older system which
incorrectly implements the pthreads signaling components. Just one
example of this is the old version of pthreads on Linux that had some
very quirky behavior when dealing with signals and thread. In particular
it would seem to chose a single thread to send ALL signals to for the
lifetime of the parent process with evidently no regard to signal masks.


8.13 A short example of pthreads and catching signals synchronously

#define _POSIX_C_SOURCE 200112L
#define NUM_LOOPS 5

#include <pthread.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

int main ( int argc, char ** argv )
{
   int count;
   int result;

   sigset_t mask;
   siginfo_t info;
   struct timespec timeout;

   /* Create a mask holding only SIGINT - ^C Interrupt */

   sigemptyset( & mask );
   sigaddset( & mask, SIGINT );

   /* Set the mask for our main thread to include SIGINT */

   pthread_sigmask( SIG_BLOCK, & mask, NULL );

   /* Max wait for ^C set to 5 seconds */

   timeout.tv_sec = 5;
   timeout.tv_nsec = 0;

   /*
      Receive max combination of NUM_LOOP signals
      or timeouts from sigtimedwait
   */

   for ( count = 0; count < NUM_LOOPS; ++ count )
   {
     result = sigtimedwait( & mask, & info, & timeout );

     if ( -1 == result && EAGAIN == errno )
     {
       printf( "Timeout occurred in sigtimedwait( ) call.\n" );
     }
     else
     {
       switch ( info.si_signo )
       {
         case SIGINT:
           printf( "Caught SIGINT in sigtimedwait( )\n" );
         break;

         /* Should never really get to default */

         default:
           printf(" Caught unexpected signal %i\n", info.si_signo );
         break;
       }
     }
   }

   return 0;
}




9. POSIX SPIN LOCKS

A spin lock is another type of mutual exclusion lock. To see the
differences between a spin lock and mutex lets look at the sequence for
mutex lock acquisition :

* A thread attempts to gain a lock on the mutex.
* If the mutex IS NOT held by another thread then the current thread is
  given the lock.
* If the mutex IS held by another thread then the current thread is
  suspended by the scheduler and resumed only when it can successfully
  gain the lock.

On the other hand, acquiring a spin lock follows this sequence:

* A thread attempts to gain a lock on the spin lock.
* If the spin lock IS NOT held by another thread then the current
  thread is given the lock.
* If the spin lock IS held by another thread then the current thread
  will go into a tight loop and in each loop iteration attempt to gain
  a lock on the spin lock.

The result of the "spin" while trying to gain a lock on a spin lock can
help provide a faster acquistion of the lock when it is released. This
is due to the fact that the context switch required when suspending and
resuming a thread that is trying to gain a lock on a regular mutex
which is held by another thread is no longer required.

There is a downside to using spin locks. Since the calling thread can
spin when trying to acquire the lock, that means it will be consuming
CPU cycles without actually performing any work. It is even possible
for such a spin to consume more cycles than the amount that would be
consumed doing a context switch using a normal mutex.


9.1 How do I determine if my system supports spinlocks?

To determine if a system implements spin locks you can check the
unistd.h system header file and check if _POSIX_SPIN_LOCKS is defined.
If it defined and the value is greater than zero, then spin locks are
provided and if the value is 200112L then the implementation should be
fully compliant with the most recent specification.


9.2 How do I create a spinlock?

A mutex is of type pthread_spinlock_t so the first step of creating an
object of this type for use is to include a declaration in your
application source code as follows:

pthread_spinlock_t [SPIN_VARIABLE_NAME];


Even though the spinlock variable has been declared and will be
instantiated on execution of the given code, it is still not ready for
use. 

Prior to the first use of the spinlock it must be initialized by
calling the function:

int pthread_spin_init
(
   pthread_spinlock_t * lock,
   int pshared
);


Where the lock parameter specifies the memory address of the spinlock we
wish to initialize and the pshared parameter specifies if the spinlock
will be accessible only by the current process or all processes on the
system. To set the latter behavior we use either the value
PTHREAD_PROCESS_PRIVATE or PTHREAD_PROCESS_SHARED respectively. Since
some systems may not provide spin locks which can be shared between
processes so you should verify that unistd.h defines
_POSIX_THREAD_PROCESS_SHARED to a value greater than zero, with the
expected value being 200112L.

On success the function will return a value of zero, the spinlock will
be initialized and in an unlocked state. If the system does not have
enough memory to allocate a init a new spinlock the error ENOMEM will
be returned while if there is some other resource is lacking an error
of EAGAIN will be returned.

Unlike mutex initialization, spinlocks do not have a static initializer
and have no additional attributes.

As with most pthread objects, once a spinlock is no longer needed it
should be destroyed to release any system resources which were
allocated to it during initialization.



9.3 How do I destroy a spinlock?

Since the a spinlock may be allocated system resources during
initialization, you should free those resources when the spinlock is no
longer required.

To do this you use the function:

int pthread_spin_destroy
(
   pthread_spinlock_t * lock
);

If the spinlock is actually still in use when this function is called
an error of EBUSY will be returned to the caller and the spinlock will
be left unmodified.



9.4 How many spinlocks can exist at the same time?

There is no specified limit to the number of spinlocks which may exist
at the same time. The primary limiting factors are memory and other
system resources. The "other system resources" are implementation
specific.



9.5 How do I lock and unlock a spinlock?

To gain an exclusive lock on a spinlock you call the function:

int pthread_spin_lock
(
   pthread_spinlock_t * lock
);

Where the lock parameter is the spinlock to which the calling thread
desires exclusive access and on success a zero is returned.

Here is another situation where a mutex and spinlock differ. For a
default "normal" type mutex, a thread attempting to lock the mutex when
it already holds it will result in a deadlock condition and the thread
never returning. For a spinlock though, attempting to gain a lock which
is already held results in the pthread_spin_lock( ) function returning
an errno of EDEADLK immediately.

Of course, as with any mutual exclusion type lock, any thread that
acquires the lock should eventually release it. The function used to
release a spinlock is:

int pthread_spin_unlock
(
   pthread_spinlock_t * lock
);

This call will return a zero if it succeeds. You can get a return value
of EBUSY if you attempt to release a spinlock that is held by a thread
other than the caller.


9.6 Can I test a spinlock to see if it is locked?

There is no way to directly test a spinlock to see if it is owned by
another thread but there is a function supplied which allows you to
attempt to acquire the spinlock and return if it cannot be acquired
immediately.

That prototype for that function is:

int pthread_spin_trylock
(
   pthread_spinlock_t * lock
);

Where lock is the spinlock that the calling thread is attempting to
acquire.

A successful call to this function will result in the calling thread
acquiring the spinlock and a value of zero being returned. If another
thread currently holds the spinlock then the function again returns
immediately but in this case with a error value of EBUSY.

There is no EDEADLK error returned as with a normal
pthread_spin_lock( ) function call where the calling thread already
owns the lock in question. So expect an EBUSY in such a case and that
you will not be able to tell if it was a result of a recursive locking
attempt or just due to another thread currently holding the lock.


9.7 Can I specify how long a thread will spin on a spinlock?

There is no POSIX function or parameter to specify how long a thread
will spin when attempting to acquire a spinlock held by another thread.
If you desire this type of functionality then possibly the best way to
do it is by creating your own spin loop in user code and utilizing the
pthread_spin_trylock.


9.8 A thread crashed that owned a spinlock, how do I unlock it?

There is no POSIX function that supplies this functionality. The
rationale, as with threads crashing that hold a mutex, is that the
critical code that the mutex was protecting will in all likelihood be
in an indeterminate state after such a crash. If that is the case then
you wouldn't want some other thread accessing that critical section.


9.9 Which thread gets spinlock ownership if several are spinning on it?

Which thread gains ownership of a spinlock when several are
simultaneously contending for it is governed mostly be the same rules
as those followed under the same conditions with mutexes.

Those basic rules are:

* looking at the scheduling policies of the spinning threads and order
them according to an implementation specific preference.

* order the threads with the prefered scheduling policy by priority.

* if there is more than one thread with the same priority then give
the lock to the one which either would get the next available time
slice on the CPU.

The third rule (especially) may not make immediate sense until you
consider the scenario where the total number of active threads in the
process exceeds the available CPUs on the system. In such it is
possible for not just one but all the threads spinning on the spinlock
to be put into a suspended state by the scheduler and to require time
slice acquisition to continue their spin.


9.10 When should I use / not use a spinlock?

The best time to use a spinlock is when the protected critical section
of code is performing a simple CPU bound task that should be completed
as quickly as possible. An example of a simple CPU bound task would be
something like incrementing counters  or perhaps reading or modifying
a memory mapped IO port.

A spinlock should NOT be used if the critical section has code in it
that is IO bound. An example of this would be waiting for keyboard
input from a user. It should also not be used to provide a lock around
code that is doing CPU bound operations which will take an extended
amount of time and cannot be used when recursive locking is required.

The reason you don't want to use a spinlock when you critical section
is performing any type of IO operations is quite simple. Such code can
block for an extremely long amount of time. That means that the
situation can arise where one thread holds the spinlock and one or more
are spinning and attempting to acquire it. The spinning threads will
not only waste CPU cycles but there is also a very good chance the
scheduler will suspend them after a while and only resume them when the
lock is released. The latter is the same thing which happens with a
regular mutex except now you have also wasted additional CPU cycles by
spinning.




10. POSIX READ-WRITE LOCKS

POSIX supplies the normal mutex type to provide complete locking on
a critical section of code. Using a normal mutex, when a thread
obtains the mutex all other threads are forced to block until that
mutex is released by the owner.

What about the situation where the vast majority of threads are simply
reading the data?  If this is the case then we should not care if
there is 1 or up to N readers in the critical section at the same
time.  In fact the only time we would normally care about exclusive
ownership is when a writer needs access to the code section. 

Fortunately, POSIX provides yet another type of mutex called the
read-write lock or rwlock that helps improve the efficiency of
such code.  A ( simplified ) example of a thread's behavior when
attempting to obtain possession of the rwlock is:

* thread is a reader
* thread attempts to enter critical section
* no thread(s) in critical section, give access
* if current thread in critical section is a writer, then block
* if current thread(s) in critical section are reader(s), give access

and:

* thread is a writer
* thread attempts to enter critical section
* no thread(s) in critical section, give access
* if current thread in critical section is a writer, then block
* if current thread(s) in critical section are reader(s), then block

There is also the case where we can end up with a combination of
readers and writers queued waiting to get ownership of the lock. Will
the readers or the writers be given preference for access, will the
threads follow a FIFO ordering or is there some other method to
determine the proper ordering?  The truth of the matter is that the
POSIX specification says little about this and it is implementation
specific.  The result is that you really need to know how the
target system implements the rwlock to know the answer.

Having said that there is one follow up.  Most implementations seem
to make the reasonable conclusion that if you are going to be employing
rwlocks then the majority of the threads will be readers.  This means
that if more than one readers and writer(s) are queued against a lock
the writer(s) will be given priority.

You should also see the section on reader starvation.



10.1 How do I determine if my system supports read-write locks?

If your system provides an implementation of POSIX threading then it
should also be providing the definitions and functions for rwlock
creation, manipulation and destruction.

If a system defines the symbolic constant _POSIX_THREADS in the file
unistd.h to a value greater than 0, it provides an implementation of
POSIX threading.



10.2 How do I create a read-write lock?

A mutex is of type pthread_mutex_t so the first step of creating an
object of this type for use is to include a declaration in your
application source code as follows:

pthread_rwlock_t [RWLOCK_VARIABLE_NAME];


Even though the rwlock object has been declared and will be
instantiated on execution of the given code, it is still not ready for
use. Prior to the first use of the rwlock it must be initialized by
calling the function:

int pthread_rwlock_init
(
  pthread_rwlock_t * restrict rwlock,
  const pthread_rwlockattr_t * restrict attr
);

Where the rwlock parameter specifies a pointer to the rwlock we are
going to initialize and the attr specifies the characteristics of the
lock. If the attr parameter is NULL, the default attributes are used
by the system. See the next few sections for details about the
pthread_rwlockattr_t type, its creation and the manipulation of the
allowed attributes.

This function can fail and return an error number in a variety of
situations.  Some examples are ENOMEM if system memory is exhausted,
EAGAIN if some other system resource is exhausted and EINVAL if an
attempt is made to init a rwlock that was previously initialized but
not yet destroyed.

On success the function will return a value of zero, the mutex will be
initialized and in an unlocked state.

Unlike a standard mutex, there is no static initializer specified by
POSIX for read-write locks. 



10.3 How many read-write locks can exist at the same time?

There is no means to determine the maximum number of rwlocks that may
exist at the same time.  

A suggestion would be to find the number of allowed mutexes and then
divide that value by two.  The reason for the recommended division of
that value is because many system implementations use two or more
lower level mutex objects to give rwlock functionality.



10.4 How do I destroy a read-write lock?

When a read-write lock is no longer needed it should be destroyed so
that any allocated system resource many be freed for reuse. The
function which provides rwlock destruction is:

int pthread_rwlock_destroy
(
   pthread_rwlock_t * rwlock
);

The function will return a zero on success. A value of EBUSY is returned
if the mutex is currently in use and a value of EINVAL if the mutex is
invalid.



10.5 What is rwlock attribute?

The rwlock attribute allows you to modify the behavior of the created
read-write lock.  The rwlock attribute data type is defined in the file
pthread.h and specified to be pthread_rwlockattr_t.



10.6 How do I destroy a rwlock attribute?

When a rwlock attribute is no longer required, it should be
explicitly destroyed to free up any resources which may have been
allocated to it. The function provided for destruction of rwlock
attribute objects is:

int pthread_rwlockattr_destroy
(
   pthread_rwlockattr_t * attr
);

Failure to destroy rwlock attribute objects which are no longer
required may result in memory leakage and erratic program execution.



10.7 What are the available rwlock attributes?

Unlike a normal mutex which has many attributes which can be set
and manipulated, a rwlock on has:

Process Shared

Details of this attribute setting is described in the next sections.



10.8 What is the rwlock Process Shared Attribute?

The default for this attribute is PTHREAD_PROCESS_PRIVATE and indicates
that only those threads within the process which created the read-write
lock are allowed to manipulate it.

On the hand, if the attribute has been set to PTHREAD_PROCESS_SHARED,
then the rwlock may have operations performed on it by any thread that
has access to the memory where the rwlock is allocated.  This is the
case even if the other threads were created and contained by a process
other than the one which created the rwlock.

It a a read write lock which has been created with the attribute
PTHREAD_PROCESS_PRIVATE is accessed from another process the result of
any such operation is undefined.



10.9 How can I determine if a system supports the rwlock Shared
Attribute?

If the symbolic constant _POSIX_THREAD_PROCESS_SHARED is defined in
unistd.h to a value greater than zero, the system provides support of
the Shared Attribute.



10.10 What functions get/set the rwlock Shared Attribute?

To query a rwlock attribute variable for the current state of its
(Process) Shared Attribute, use the function:

int pthread_rwlockattr_getpshared
(
   const pthread_rwlockattr_t * attr,
   int * pshared
);

Where the parameter attr is a pointer to the location of the rwlock
attribute object we want to query and pshared is a pointer to the
location where the obtained result, which should be equal to
PTHREAD_PROCESS_PRIVATE or PTHREAD_PROCESS_SHARED, will be placed.

To set the Process Shared Attribute of a rwlock attribute variable,
the following function is supplied:

int pthread_rwlockattr_setpshared
(
   const pthread_rwlockattr_t * attr,
   int pshared
);

Where the parameter attr is a pointer to the rwlock attribute object
you wish to modify and the parameter pshared is the value
PTHREAD_PROCESS_PRIVATE or PTHREAD_PROCESS_SHARED.

Both functions return a 0 if the operation was successful.  They may
also return an EINVAL if an invalid parameter is supplied.



10.11 How do I find out if a rwlock is process private or shared?

There is no supplied function to find out if the rwlock is process
private or shared once the rwlock has been created.  The only options
are to implicitly know this information or to save the unmodified
rwlock attribute used during the creation.



10.12 How do I lock and unlock a read-write lock?

For a thread to unlock a rwlock which it has obtained for either
reading OR writing there is one function supplied defined as
follows:

int pthread_rwlock_unlock
(
  pthread_rwlock_t * rwlock
);

Where the parameter rwlock is a rwlock which is currently being
held by the calling thread.

On success a zero is returned to the caller. On the other hand if the
rwlock parameter is invalid the function will return an EINVAL and if
it is not held by the calling thread it will return an EPERM error.

The functions to obtain a rwlock depends on whether the calling thread
will be performing a read operation or a write operation.  For a thread
which desires read only access the functions are:


int pthread_rwlock_rdlock
(
  pthread_rwlock_t * rwlock
);

int pthread_rwlock_timedrdlock
(
  pthread_rwlock_t * restrict rwlock,
  const struct timespec * restrict abs_timeout
);

int pthread_rwlock_tryrdlock
(
  pthread_rwlock_t * rwlock
);


For threads which need exclusive access so that they may safely perform
write operations, the functions are:


int pthread_rwlock_wrlock
(
  pthread_rwlock_t * rwlock
);

int pthread_rwlock_timedwrlock
(
  pthread_rwlock_t * restrict rwlock,
  const struct timespec * restrict abs_timeout
);

int pthread_rwlock_trywrlock
(
  pthread_rwlock_t * rwlock
);


These six functions are covered in more detail in the next sections.



10.13 How do I request a lock for reading on a rwlock?

To obtain a reader lock on a rwlock a thread you use the function:

int pthread_rwlock_rdlock
(
  pthread_rwlock_t * rwlock
)

Where the parameter rwlock points to the rwlock the thread is trying to
obtain. The function will not return until the thread has obtained
ownership of the indicated rwlock or an error conditions occurs.

When the function succeeds it will return a zero.  It may also fail
and return an EINVAL if the given rwlock is invalid.  Additionally if
the calling thread already owns the rwlock and the implementation does
not allow recursive reader locks then a EDEADLK will be returned to
the caller.

Last but certainly not least, the system may have an implementation
dependent value for the maximum number of readers allowed to hold the
rwlock at the same time.  If this value is exceeded then it is also
considered an error condition and the return value will be EAGAIN.



10.14 Can I limit the time waiting for a read lock on a rwlock?

If the symbolic constant _POSIX_TIMEOUTS is defined in unistd.h to a
value greater than 0 (preferably 200112L) then the system provides the
following function:

Code:

int pthread_rwlock_timedrdlock
(
  pthread_rwlock_t * restrict rwlock,
  const struct timespec * restrict abs_timeout
);

This function is identical in behavior to pthread_rwlock_rdlock( )
except that when the calling thread would block it will then wait only
until the time specified by abs_timeout to gain ownership. If the
expiration time is reached without the thread being allowed read
access, then the function will return the error ETIMEDOUT.

Note: How abs_timeout is interpreted depends on the resolution of the
clock used by the system. From experience and due to the widely varying
resolutions employed on available systems, I personally do not rely on
any granularity higher than second intervals.



10.15 Can I check if a rwlock would block a reader thread?

To test if a thread wanting reader access to a rwlock would block you
can use the function:

Code:

int pthread_rwlock_tryrdlock
(
  pthread_rwlock_t * rwlock
);


If the calling thread can immediately obtain the lock then it will do
so and the function will return a value of zero. If the calling thread
would block or it already owns the rwlock then this function will
return a value of EBUSY. It can also return an EINVAL if the rwlock
parameter is invalid for any reason.

It should also be mentioned that like pthread_rwlock_rdlock, this
function can return EAGAIN if the implementation dependent value for
the maximum number allowed readers holding the lock has already been
reached.



10.16 How do I request a lock for writing on a rwlock?

To obtain a writer lock on a rwlock a thread you use the function:

int pthread_rwlock_wrlock
(
  pthread_rwlock_t * rwlock
)

Where the parameter rwlock points to the rwlock the thread is trying to
obtain. The function will not return until the thread has obtained
ownership of the indicated rwlock or an error conditions occurs.

When the function succeeds it will return a zero.  It may also fail
and return an EINVAL if the given rwlock is invalid.  Additionally if
the calling thread already owns the rwlock a EDEADLK will be returned
to the caller.

Since only one writer may own the rwlock at one time this function will
never return a EAGAIN.  This behavior is different from what is can
happen during reader lock calls with the pthread_rwlock_rdlock and
associated functions.



10.17 Can I check if a rwlock would block a writer thread?

To test if a thread wanting writer access to a rwlock would block you
can use the function:

Code:

int pthread_rwlock_trywrlock
(
  pthread_rwlock_t * rwlock
);


If the calling thread can immediately obtain the lock then it will do
so and the function will return a value of zero. If the calling thread
would block or it already owns the rwlock then this function will
return a value of EBUSY. It can also return an EINVAL if the rwlock
parameter is invalid for any reason.



10.18 Can I limit the time waiting for a write lock on a rwlock?

If the symbolic constant _POSIX_TIMEOUTS is defined in unistd.h to a
value greater than 0 (preferably 200112L) then the system provides the
following function:

Code:

int pthread_rwlock_timedwrlock
(
  pthread_rwlock_t * restrict rwlock,
  const struct timespec * restrict abs_timeout
);

This function is identical in behavior to pthread_rwlock_wrlock( )
except that when the calling thread would block it will then wait only
until the time specified by abs_timeout to gain ownership. If the
expiration time is reached without the thread being allowed write
access, then the function will return the error ETIMEDOUT.

Note: How abs_timeout is interpreted depends on the resolution of the
clock used by the system. From experience and due to the widely varying
resolutions employed on available systems, I personally do not rely on
any granularity higher than second intervals.



10.19 A thread crashed that owned a read-write lock, how do I unlock
  it?

While a read-write lock acts like a mutex in many ways there is a bit
of a difference in behavior when a thread crashes that "holds" 
the lock.  That difference is that under some conditions threads may
still be able to gain ownership of the rwlock.

The five basic behaviors you can typically expect are:

* If a writer crashes while holding the thread it is permanently
    locked and no other threads will be able to obtain it.

* If a reader crashes then usually, as long as only readers attempt
    to obtain the lock afterward, they will probably succeed.

* If a reader crashes and a writer attempts to gain the lock, then
    writers will always permanently permanently.

* If the rwlock gives preference to readers and a reader crashes
    while holding the lock then all subsequent writers will always
    block but it is implementation specific if future readers will
    be given access.

* If the rwlock gives preference to writers and a reader crashes
    while holding the lock then once a writer attempts to gain
    the lock then that writer and all subsequent readers and
    writers will block.

The descriptions I have given are a bit "loose" but do provide a
general idea of what you can expect.  Again, a lot of this behavior
is implementation specific.

It should be mentioned that since the possibility exists for a reader
to crash but future readers to still gain access to the critical
section, a rwlock can be a lot more dangerous to use than a standard
mutex.

There is a rational behind that previous statement. Even thought the
crashed thread may have been a reader, which means it promised not
to change data in the critical section, the effect of an uncontrolled
crash may cause just such a modification.  The end result is that your
program could potentially continue execution under these conditions 
even though the data is in an indeterminant state.



10.20 Why are my reader threads suffering from starvation?

As was mentioned in the section introduction, most read-write lock
implementations give preference to writers over readers.  This means
that if writers are trying to gain the lock at such a high frequency
that there is always at least one writer is blocked and waiting for
ownership then any incoming readers will never even have the chance
to gain ownership.  The end result is, of course, slow starvation of
the readers.

Of course, you system could also for some reason give preference to
readers instead.  This situation is unlikely but not impossible. If it
is the case and your reader threads are bombarding the rwlock then the
writers may be starved.

If you run into either of these situations then the only real answer
is that the rwlock is not appropriate and you should probably modify
your code to use a regular mutex.



10.21 When should I use / not use a read-write lock?

The general things I look for when determining if a read-write lock
would be more appropriate than a standard mutex are:

* the rwlock implementation gives preference to writers ( normal )
  and twenty-five percent or less of the expected operations require
  write locks.

* the rwlock implementation gives preference to readers ( unusual )
  and fifty percent or less of the expected operations require
  reader locks.

* clusters of either same type requests ( read or write ) occur
  together with a large interval of time passing between the clusters.

The last item may seem unusual but the premise is that if a burst of
all writer operations occur then the rwlock will act like a regular
mutex.  If a burst of readers come thru then you will get the benefit
of all the threads being allowed into the critical section
concurrently.  The reason for requiring the time interval is because
if there wasn't adequate spacing between reader and writer bursts you
could end up with thread starvation as discussed in a previous section.