步子百科步子百科

限流是什么意思(Nginx)

1.安装依赖环境yum -y install wget gcc-c++ ncurses ncurses-devel cmake make perl bison openssl openssl-devel gcc* libxml2 libxml2-devel curl-devel libjpeg* libpng* freetype* autoconf automake zlib* fiex* libxml* libmcrypt* libtool-ltdl-devel* libaio libaio-devel bzr libtool2.安装opensslwget https://www.openssl.org/source/openssl-1.0.2s.tar.gztar -zxvf openssl-1.0.2s.tar.gzcd /usr/local/src/openssl-1.0.2s./config --prefix=/usr/local/openssl-1.0.2smakemake install3.安装pcrewget https://ftp.pcre.org/pub/pcre/pcre-8.43.tar.gztar -zxvf pcre-8.43.tar.gzcd /usr/local/src/pcre-8.43./configure --prefix=/usr/local/pcre-8.43makemake install4.安装zlibwget https://sourceforge.net/projects/libpng/files/zlib/1.2.11/zlib-1.2.11.tar.gztar -zxvf zlib-1.2.11.tar.gzcd /usr/local/src/zlib-1.2.11./configure --prefix=/usr/local/zlib-1.2.11makemake5.安装Nginxwget http://nginx.org/download/nginx-1.17.2.tar.gztar -zxvf nginx-1.17.2.tar.gzcd /usr/local/src/nginx-1.17.2./configure --prefix=/usr/local/nginx-1.17.2 --with-openssl=/usr/local/src/openssl-1.0.2s --with-pcre=/usr/local/src/pcre-8.43 --with-zlib=/usr/local/src/zlib-1.2.11 --with-http_ssl_modulemakemake install

这里需要注意的限流是:安装Nginx时,指定的什意思是openssl、pcre和zlib的限流源码解压目录,安装完成后Nginx配置文件的什意思完整路径为:/usr/local/nginx-1.17.2/conf/nginx.conf。

Nginx负载均衡配置1.负载均衡配置http { …… upstream real_server { server 192.168.103.100:2001 weight=1; #轮询服务器和访问权重 server 192.168.103.100:2002 weight=2; } server { listen 80; location / { proxy_pass http://real_server; } }}2.失败重试配置upstream real_server { server 192.168.103.100:2001 weight=1 max_fails=2 fail_timeout=60s; server 192.168.103.100:2002 weight=2 max_fails=2 fail_timeout=60s;}

意思是限流在fail_timeout时间内失败了max_fails次请求后,则认为该上游服务器不可用,什意思然后将该服务地址踢除掉。限流fail_timeout时间后会再次将该服务器加入存活列表,什意思进行重试。限流

Nginx限流配置1.配置参数

limit_req_zone指令设置参数

limit_req_zone $binary_remote_addr zone=mylimit:10m rate=10r/s;limit_req_zone定义在http块中,什意思$binary_remote_addr表示保存客户端IP地址的限流二进制形式。Zone定义IP状态及URL访问频率的什意思共享内存区域。zone=keyword标识区域的限流名字,以及冒号后面跟区域大小。什意思16000个IP地址的限流状态信息约1MB,所以示例中区域可以存储160000个IP地址。Rate定义最大请求速率。示例中速率不能超过每秒10个请求。2.设置限流location / { limit_req zone=mylimit burst=20 nodelay; proxy_pass http://real_server;}

burst排队大小,nodelay不限制单个请求间的时间。

3.不限流白名单geo $limit { default 1;192.168.2.0/24 0;}map $limit $limit_key { 1 $binary_remote_addr;0 "";}limit_req_zone $limit_key zone=mylimit:10m rate=1r/s;location / { limit_req zone=mylimit burst=1 nodelay; proxy_pass http://real_server;}

上述配置中,192.168.2.0/24网段的IP访问是不限流的,其他限流。

IP后面的数字含义:

24表示子网掩码:255.255.255.016表示子网掩码:255.255.0.08表示子网掩码:255.0.0.0

Nginx缓存配置1.浏览器缓存

静态资源缓存用expire

location ~* .(jpg|jpeg|png|gif|ico|css|js)$ { expires 2d;}

Response Header中添加了Expires和Cache-Control,

静态资源包括(一般缓存)

普通不变的图像,如logo,图标等js、css静态文件可下载的内容,媒体文件

协商缓存(add_header ETag/Last-Modified value)

HTML文件经常替换的图片经常修改的js、css文件基本不变的API接口

不需要缓存

用户隐私等敏感数据经常改变的api数据接口2.代理层缓存//缓存路径,inactive表示缓存的时间,到期之后将会把缓存清理proxy_cache_path /data/cache/nginx/ levels=1:2 keys_zone=cache:512m inactive = 1d max_size=8g;location / { location ~ \.(htm|html)?$ { proxy_cache cache; proxy_cache_key $uri$is_args$args; //以此变量值做HASH,作为KEY //HTTP响应首部可以看到X-Cache字段,内容可以有HIT,MISS,EXPIRES等等 add_header X-Cache $upstream_cache_status; proxy_cache_valid 200 10m; proxy_cache_valid any 1m; proxy_pass http://real_server; proxy_redirect off; } location ~ .*\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$ { root /data/webapps/edc; expires 3d; add_header Static Nginx-Proxy; }}

在本地磁盘创建一个文件目录,根据设置,将请求的资源以K-V形式缓存在此目录当中,KEY需要自己定义(这里用的是url的hash值),同时可以根据需要指定某内容的缓存时长,比如状态码为200缓存10分钟,状态码为301,302的缓存5分钟,其他所有内容缓存1分钟等等。可以通过purger的功能清理缓存。

AB测试/个性化需求时应禁用掉浏览器缓存。

Nginx黑名单1.一般配置location / { deny 192.168.1.1; deny 192.168.1.0/24; allow 10.1.1.0/16; allow 2001:0db8::/32; deny all;}2. Lua+Redis动态黑名单(OpenResty)

安装运行

yum install yum-utilsyum-config-manager --add-repo https://openresty.org/package/centos/openresty.repoyum install openrestyyum install openresty-resty查看yum --disablerepo="*" --enablerepo="openresty" list available运行service openresty start

配置(/usr/local/openresty/nginx/conf/nginx.conf)

lua_shared_dict ip_blacklist 1m;server { listen 80; location / { access_by_lua_file lua/ip_blacklist.lua; proxy_pass http://real_server; }}

lua脚本(ip_blacklist.lua)

local redis_host = "192.168.1.132"local redis_port = 6379local redis_pwd = 123456local redis_db = 2-- connection timeout for redis in ms.local redis_connection_timeout = 100-- a set key for blacklist entrieslocal redis_key = "ip_blacklist"-- cache lookups for this many secondslocal cache_ttl = 60-- end configurationlocal ip = ngx.var.remote_addrlocal ip_blacklist = ngx.shared.ip_blacklistlocal last_update_time = ip_blacklist:get("last_update_time");-- update ip_blacklist from Redis every cache_ttl seconds:if last_update_time == nil or last_update_time < ( ngx.now() - cache_ttl ) then local redis = require "resty.redis"; local red = redis:new(); red:set_timeout(redis_connect_timeout); local ok, err = red:connect(redis_host, redis_port); if not ok then ngx.log(ngx.ERR, "Redis connection error while connect: " .. err); else local ok, err = red:auth(redis_pwd) if not ok then ngx.log(ngx.ERR, "Redis password error while auth: " .. err); else local new_ip_blacklist, err = red:smembers(redis_key); if err then ngx.log(ngx.ERR, "Redis read error while retrieving ip_blacklist: " .. err); else ngx.log(ngx.ERR, "Get data success:" .. new_ip_blacklist) -- replace the locally stored ip_blacklist with the updated values: ip_blacklist:flush_all(); for index, banned_ip in ipairs(new_ip_blacklist) do ip_blacklist:set(banned_ip, true); end -- update time ip_blacklist:set("last_update_time", ngx.now()); end end endendif ip_blacklist:get(ip) then ngx.log(ngx.ERR, "Banned IP detected and refused access: " .. ip); return ngx.exit(ngx.HTTP_FORBIDDEN);end

Nginx灰度发布1.根据Cookie实现灰度发布

根据Cookie查询version值,如果该version值为v1转发到host1,为v2转发到host2,都不匹配的情况下转发到默认配置。

upstream host1 { server 192.168.2.46:2001 weight=1; #轮询服务器和访问权重 server 192.168.2.46:2002 weight=2;}upstream host2 { server 192.168.1.155:1111 max_fails=1 fail_timeout=60;}upstream default { server 192.168.1.153:1111 max_fails=1 fail_timeout=60;}map $COOKIE_version $group { ~*v1$ host1; ~*v2$ host2; default default;}lua_shared_dict ip_blacklist 1m;server { listen 80; #set $group "default"; #if ($http_cookie ~* "version=v1"){ # set $group host1; #} #if ($http_cookie ~* "version=v2"){ # set $group host2; #} location / { access_by_lua_file lua/ip_blacklist.lua; proxy_pass http://$group; }}2.根据来路IP实现灰度发布server { …………… set $group default; if ($remote_addr ~ "192.168.119.1") { set $group host1; } if ($remote_addr ~ "192.168.119.2") { set $group host2; }3.更细粒度灰度发布

参考:

https://github.com/sunshinelyz/ABTestingGateway