Next
Previous Contents
LVS-HOWTO
Joseph Mack (C) 1999-2001, released under GPL
jmack@wm7d.net
v1.4, May 2001
Install, testing and running of a Linux Virtual Server with 2.2.x and 2.4.x kernels
1.
Introduction
1.1 ChangeLog
1.2 Purpose of this HOWTO
1.3 Nomenclature/Abbreviations
1.4 What is an LVS?
1.5 Minimal knowledge required
1.6 LVS Failure
1.7 Thanks
1.8 Mailing list, subscribing, unsubscribing and searchable archives
1.9 getting technical help
1.10 Posting problems/questions to the mailing list
1.11 ToDo List
1.12 Other load balancing solutions
1.13 Software/Information useful/related to LVS
2.
Getting Files
2.1 Director Code
2.2 RealServer Code
2.3 Configure Script
3.
The arp Problem
3.1 The problem
3.2 The cure(s)
3.3 The ARP problem, the first inklings
3.4 A posting to the mailinglist by Peter Kese
3.5 random mailings on the arp problem
3.6 Is the arp behaviour of 2.2.x kernel a bug?
3.7 How to tell if an interface is replying to arp requests
3.8 Arp caching defeats Heartbeat switchover
3.9 More on the arp problem
3.10 Properties of devices for the VIP
3.11 Topologies for VS-DR and VS-Tun LVS's
3.12 A discussion about the arp problem
3.13 ATM/ethernet and router problems
4.
Collect Hardware
4.1 minimum setup
4.2 Gotchas
4.3 Test with telnet (or netcat)
5.
Choose LVS Forwarding Type
5.1 Comparison of VS-NAT, VS-DR and VS-Tun
5.2 Expected LVS performance
5.3 Initial setup steps
6.
Install - General
6.1 Director
6.2 Real-servers
6.3 iptables/ipchains compatability problems
7.
Ipvsadm
7.1 Using ipvsadm
7.2 Compile ipvsadm for each new ipvs
7.3 schedulers
7.4 does rr equally distribute the load?
7.5 persistent connections
7.6 Changing weights with ipvsadm
7.7 experimental scheduling code
8.
Fwmarks
8.1 Introduction
8.2 single port service: telnet with fwmarks
8.3 Grouping services: single group, active ftp(20,21)
8.4 Grouping services: two groups, active ftp(20,21) and e-commerce(80,443)
8.5 passive ftp
8.6 fwmark with VS-NAT
8.7 Collisions between fwmark and VIP rules
8.8 persistence granularity with fwmark
8.9 fwmark allows VS-DR director to be default gw for real-servers
8.10 Routing to director and real-servers in an LVS setup with fwmark
8.11 fwmark simplifies configuration for large numbers of addresses
8.12 Example: firewall farm
8.13 Example: LVS'ing a CIDR block
8.14 Example: forwarding based on client source IP
8.15 Example: load balancing multiple class C networks
8.16 Example: proxy server
8.17 Example: transparent web cache
8.18 Example: Dynamically generated images in webpages
8.19 Appendix 1: Specificiations for grouping of services with fwmarks
8.20 Appendix 2: Demonstration of grouping services with fwmarks
8.21 Appendix 3: Announcement of grouping services with fwmarks
9.
Configure tools
9.1 Configure
10.
Services
10.1 setting up a new service
10.2 services must be setup for forwarding type
10.3 ftp general
10.4 ftp (active) - the classic command line ftp
10.5 ftp (passive)
10.6 ftp is difficult to secure
10.7 evaluation of SuSE ftp proxy
10.8 telnet
10.9 ssh
10.10 dns
10.11 sendmail/smtp/pop3/qmail
10.12 authd/identd (port 113) and tcpwrappers (tcpd)
10.13 http name and IP-based (with VS-DR or VS-Tun)
10.14 http with VS-NAT
10.15 httpd normally closes connections
10.16 Persistence with http; browser opens many connections to httpd
10.17 Dynamically generated images on web pages
10.18 other considerations with http: logs, shutting down httpd, cookies, mod_proxy, indexing programs
10.19 https
10.20 Databases
10.21 Cookies
10.22 r commands; rsh, rcp, and their ssh replacements
10.23 nfs
10.24 RealNetworks streaming protocols
11.
VS-NAT
11.1 Introduction
11.2 Example Two Network VS-NAT (VIP and RIPs on different network)
11.3 All packets from the real-server to the outside world must go through the director
11.4 Run configure
11.5 How VS-NAT works
11.6 In VS-NAT, how do packets get back to the client, or how does the
11.7 Performance of VS-NAT
11.8 One network VS-NAT
11.9 Various debugging techniques for routes
11.10 Postings from the mailing list
12.
VS-DR
12.1 How VS-DR works
12.2 Handling the arp problem for VS-DR
12.3 VS-DR scales well
12.4 VS-DR director is default gw for real-servers
12.5 default gw(s) and routing with VS-DR/VS-Tun
12.6 routing to real-server from director
13.
VS-Tun
13.1 How VS-Tun works
13.2 Configure VS-Tun
13.3 Real-servers on different network(s) to director
13.4 VS-Tun Questions
14.
localnode
15.
Transparent proxy (TP or Horms' method)
15.1 General
15.2 How you use TP
15.3 Transparent proxy for 2.4.x
15.4 Transparent proxy Q&A
15.5 Identd doesn't delay connection with when packets are received by TP on 2.4.x real-servers
15.6 What IP TP packets arriving on?
15.7 Take home lesson for setting up TP on real-servers
15.8 Handling identd requests from 2.4.x VS-DR real-servers using TP
15.9 Performance of Transparent Proxy
16.
Authd/Identd
16.1 What is authd/identd?
16.2 comp.os.linux.security FAQ on identd
16.3 Why identd is a problem for LVS
16.4 tcpdumps of connections delayed by identd
16.5 There are solutions to identd problem in some cases
16.6 Turn off tcpwrappers
16.7 Identd and smtp/pop/qmail
17.
Squid Real-Servers (poor man's L7 switch)
17.1 Terminology
17.2 Preview
17.3 Let's start assembling
17.4 One squid
17.5 Another squid
17.6 Combining pieces with LVS
17.7 Problems
18.
Details of LVS operation
18.1 Director Hash Table
18.2 Port range limitations
18.3 DoS
18.4 Active/Inactive connnection
18.5 Creating large numbers of InActConn with testlvs; testing DoS strategies
18.6 Debugging LVS
18.7 Security Issues
18.8 MTU discovery
18.9 ICMP handling
18.10 Filesystems for real-server content: the many reader, single writer problem
19.
Failover protection
19.1 Director failure
19.2 Real-server failure
19.3 Service/real-server failout
19.4 Mon for server/service failout
19.5 BIG CAVEAT
19.6 About Mon
19.7 Mon Install
19.8 Mon Configure
19.9 Testing mon without LVS
19.10 Can virtualserver.alert send commands to LVS?
19.11 Running mon with LVS
19.12 Why is the LVS monitored for failures by an external agent rather than by the kernel?
20.
Misc/FAQ/Wisdom from the mailing list
20.1 Setting up an LVS with inetd
20.2 How to bring down a real-server for maintenance (eg swap disks)
20.3 Howto turn your single node ftp/http server into an LVS without taking it off-line
20.4 Other projects like LVS - Beowulf
20.5 Projects like LVS - Eddie
20.6 Troubles with tulip cards
20.7 About PPC (persistent port connection) (for 2.2.12 kernels)
20.8 Related to PPC - Sticky connections
20.9 Thundering herd problem, when down machine(s) come on line
20.10 on the need for extended testing
20.11 loopback on Solaris
20.12 Having one director handling multiple LVS sites
20.13 Running multiple directors (each with their own IP)
20.14 Running clients (eg telnet) on real-servers
20.15 Setting up NAT clients on VS-DR real-servers
20.16 Timeouts
20.17 tcpdump
20.18 Bringing down aliased devices
20.19 Backing up real-servers/keeping in sync
20.20 Malicious attacks (SYN floods)
20.21 Does SMP help on the director?
20.22 Multiple IPs on the Director
20.23 Expanding port number range
20.24 Performance Hints from the Squid people
20.25 Problems with large uptimes
20.26 Testimonials
20.27 Transport Layer Security(TLS)
20.28 rcp and friends on LVS (better to use ssh)
20.29 Forwarding an httpd request based on file name not load (mod_proxy)
20.30 URL parsing
20.31 can I run my ipchains firewall and LVS on the same box?
20.32 Setting up a hot spare server
21.
Useful things that have other place (yet)
21.1 Files which are kernel version dependant eg System.map and ipvsadm
21.2 Ramdisk
21.3 cscope
22.
FAQ
22.1 Help! My LVS doesn't work
22.2 My LVS doesn't work: ipvsadm shows entries in InActConn, but none in ActiveConn
22.3 initial connection is delayed, but once connected everything is fine
22.4 How fast/big should my director be
22.5 Does the director handle ICMP?
22.6 I get "connection refused" from the client
22.7 Any recommendations for a NIC?
22.8 Does SMP help?
Next
Previous Contents