LINUX VS SOLARIS Performance

Table : Some Libraries in Linux and Solaris OS

Solaris OS	Linux	Description
libc	libc	The standard C library (POSIX, SysV, ANSI, etc.) See man libc on Solaris OS.
libucb	libc	UCB (University California Berkeley) compatibility library
libmalloc	libc	There are several different malloc libraries; the default is in libc.
libsocket	libc	Socket library (sockets are in libc on Linux).
libxnet	libc	X/Open Networking library
libresolv	libresolv	DNS routines (and on Solaris OS, inet_* routines)
libnsl	libnsl/libc	Network services library (linux - nis/nis+ routines)
librpc	librpc	RPC functions
libslp	libslp	Service Location Protocol
libsasl	libsasl	Simple Authentication and Security Layer
libaio	libaio	Asynchronous I/O library
libdoor		Door support ( door_create(), door_return(), etc.)
librt	librt	POSIX Real Time library
libcfgadm		Configuration administration library
libcontract		Contract management library (see man contract.4 on Solaris OS)
libcpc		CPU performance counter library (on Linux, may need to install kernel module?)
libdat		Direct Access Transport Library (see http://www.datcollaborative.org)
libelf	libelf	ELF support library
libm	libm	Math library

LINUX VS SOLARIS PERFORMANCE

Linux : orange color

Solaris : blue color

The next sections take a closer look at some of the system calls and libraries. We'll concentrate on what's different between the systems.

One of these is Linux, the other SmartOS. Same hardware:

• One system is 14% slower

• Imagine that's your system – you'd want to know why

• I recently had a customer with a complex performance issue try

a one-liner like this, as a simple test, and with a similar result.

It's an interesting tour of some system differences

systemA$ time perl -e 'for ($i = 0; $i < 100_000_000; $i++) { $s = "SCaLE12x" }'

real 0m18.534s

user 0m18.450s

sys 0m0.018s

systemB$ time perl -e 'for ($i = 0; $i < 100_000_000; $i++) { $s = "SCaLE12x" }'

real 0m16.253s

user 0m16.230s

sys 0m0.010s

Can the kernel make a difference? ... As a reminder:

• The program makes no system calls during the loop

• Yes, for a number of reasons:

• Setting the string involves memory I/O, and the kernel

controls memory placement. Allocating nearby memory in

a NUMA system can significantly improve performance

• The kernel may also control the CPU clock speed (eg, Intel

SpeedStep), and vary it for temp or power reasons

• The program could be perturbed by interrupts: eg, network

I/O (although the performance effect should be small).

During a perturbation, the kernel CPU scheduler may

migrate the thread to another CPU, which can hurt

performance (cold caches, memory locality)

Big Differences: Linux

Up-to-date packages Latest application versions, with the latest

performance fixes

Large community Weird perf issue? May be answered on

stackoverflow, or discussed at meetups

More device drivers There can be better coverage for high

performing network cards or driver features

futex Fast user-space mutex

RCU Fast-performing read-copy updates

btrfs Modern file system with pooled storage

DynTicks Dynamic ticks: tickless kernel, reduces

interrupts and saves power

SLUB Simplified version of SLAB kernel memory

allocator, improving performance

System Similarities

• It's important to note that many performance-related features

are roughly equivalent:

• Both are Unix-like systems: processes, kernel, syscalls,

time sharing, preemption, virtual memory, paged virtual

memory, demand paging, ...

• Similar modern features: unified buffer cache, memory

mapped files, multiprocessor support, CPU scheduling

classes, CPU sets, 64-bit support, memory locality,

resource controls, PIC profiler, epoll, ...

What Solaris can learn from Linux performance?

• Packaging

• Community

• Compiler Options

• likely()/unlikely()

• Tickless Kernel

• Process Swapping

• Either learning what to do, or learning what not to do...

• Overcommit & OOM Killer

• SLUB

• Lazy TLB

• TIME_WAIT Recycling

• sar

• KVM

What Linux can learn from Solaris performance?

• ZFS

• Zones

• STREAMS

• Symbols

• prstat -mLc

• vfsstat

• DTrace

• Culture

Lead Performance Engineer

brendan@joyent.com

Brendan Gregg

@brendangregg