Nginx Ingress 由资源对象 Ingress、Ingress 控制器、Nginx 三部分组成,Ingress 控制器用以将 Ingress 资源实例组装成 Nginx 配置文件(nginx.conf),并重新加载 Nginx 使变更的配置生效。当它监听到 Service 中 Pod 变化时通过动态变更的方式实现 Nginx 上游服务器组配置的变更,无须重新加载 Nginx 进程。
修改nginx.conf有三种方式,1,直接修改nginx.conf,2,configMap;3,Ingress;其中configMap可以作为基础模板,它能改动的地方更多,分为下面4个部分
- main-snippet string "" 在 main 指令域添加 Nginx 配置指令
- http-snippet string "" 在 http 指令域添加 Nginx 配置指令
- server-snippet string "" 在 server 指令域添加 Nginx 配置指令
- location-snippet string "" 在 location 指令域添加 Nginx 配置指令
而Ingress只能修改server和location域的指令,具体是通过annotations:
- nginx.ingress.kubernetes.io/server-snippet string 在 server 指令域添加 Nginx 配置指令
- nginx.ingress.kubernetes.io/configuration-snippet string 在 location 指令域添加 Nginx 配置指令
首选我们搭建openresty环境,通过直接修改nginx.conf的方式来用lua连接mysql
代码语言:javascript复制helm install ingress-nginx bitnami/nginx-ingress-controller
NAME: ingress-nginx
LAST DEPLOYED: Fri Jun 3 22:29:03 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
把镜像更新一下
代码语言:javascript复制% kubectl -n nginx-ingress edit deploy nginx-ingress-nginx-ingress-controller
image: docker.io/bitnami/nginx-ingress-controller:latest
% kubectl -n nginx-ingress edit deployment.apps/nginx-ingress-nginx-ingress-controller-default-backend
image: docker.io/bitnami/openresty:latest
修改nginx连接方式为nodePort
代码语言:javascript复制 % kubectl -n nginx-ingress edit svc nginx-ingress-nginx-ingress-controller-default-backend
service/nginx-ingress-nginx-ingress-controller-default-backend edited
type: NodePort
测试下
代码语言:javascript复制 % curl http://127.0.0.1:31307/
openresty已经起来了,但是由于配置文件在pod里面,因此我们需要挂载一个volume,把配置文件映射到主机上
代码语言:javascript复制apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-volume
spec:
storageClassName: hostpath
capacity:
storage: 200Mi
accessModes:
- ReadWriteOnce
hostPath:
path: "/Users/xiazemin/go/src/github.com/xiazemin/k8s_learn/ingress/lua/"
代码语言:javascript复制% kubectl apply -f ingress/lua/pv.yaml
persistentvolume/pv-volume created
代码语言:javascript复制kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-volume
spec:
storageClassName: hostpath
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Mi
代码语言:javascript复制 % kubectl apply -f ingress/lua/pvc.yaml
persistentvolumeclaim/pvc-volume created
在部署中映射主机目录
代码语言:javascript复制 volumeMounts:
- name: pv-volume
mountPath: /opt/bitnami/openresty/nginx/conf/server_blocks/
volumes:
- name: pv-volume
persistentVolumeClaim:
claimName: pvc-volume
% kubectl -n nginx-ingress edit deployment.apps/nginx-ingress-nginx-ingress-controller-default-backend
注意容器内的路径使用
代码语言:javascript复制/opt/bitnami/openresty/nginx/conf/server_blocks/
这个bitnami的helm chart上抽象好的nginx配置文件加载路径,否则需要更改docker镜像,设置环境变量,巨麻烦。
修改nginx.conf文件
代码语言:javascript复制# Based on https://www.nginx.com/resources/wiki/start/topics/examples/full/#nginx-conf
# you do not need the following line if you are using
# the ngx_openresty bundle:
server {
# Port to listen on, can also be set in IP:PORT format
listen 8080;
location / {
stub_status on;
access_log on;
content_by_lua_block {
ngx.say("hello lua");
}
}
}
重启pod,测试下
代码语言:javascript复制 % curl http://localhost:31307/
hello lua
说明基础环境已经成功了,下面我们连接mysql,首先部署mysql
代码语言:javascript复制% helm install mysql bitnami/mysql -n nginx-ingress
NAME: mysql
LAST DEPLOYED: Sun Jun 5 13:31:57 2022
NAMESPACE: nginx-ingress
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
** Please be patient while the chart is being deployed **
Tip:
修改nginx.conf支持加载lua文件
代码语言:javascript复制# Based on https://www.nginx.com/resources/wiki/start/topics/examples/full/#nginx-conf
# you do not need the following line if you are using
# the ngx_openresty bundle:
lua_package_path "/opt/bitnami/openresty/nginx/conf/server_blocks/rocks-5.1/lua-resty-mysql/lib/?.lua;;";
server {
lua_code_cache off;
# Port to listen on, can also be set in IP:PORT format
listen 8080;
location /lua {
resolver kube-dns.kube-system.svc.cluster.local;
access_log on;
content_by_lua_file /opt/bitnami/openresty/nginx/conf/server_blocks/hello.lua;
header_filter_by_lua_file /opt/bitnami/openresty/nginx/conf/server_blocks/header.lua;
}
location / {
resolver kube-dns.kube-system.svc.cluster.local;
stub_status on;
access_log on;
content_by_lua_block {
ngx.say("hello lua");
}
}
}
关闭lua的代码缓冲,可以方便热更新,不用重启nginx
代码语言:javascript复制 lua_code_cache off;
指定lua包的路径
代码语言:javascript复制lua_package_path "/opt/bitnami/openresty/nginx/conf/server_blocks/rocks-5.1/lua-resty-mysql/lib/?.lua;;";
lua-resty-mysql 是通过包管理工具luarocks下载的
代码语言:javascript复制 % luarocks --lua-dir=/usr/local/opt/lua@5.1 install lua-resty-mysql 0.15-0
安装成功后把路径链接到我们映射的目录里面,我们在代码里面就可以直接引用了。
注意nginx需要在location里指定域名解析器,否则没法访问我们内部的service
代码语言:javascript复制 resolver kube-dns.kube-system.svc.cluster.local;
下面是我们修改内容和header的两段代码
代码语言:javascript复制ngx.say("hello lua file test");
ngx.say(ngx.config.ngx_lua_version);
local mysql = require "resty.mysql"
local db, err = mysql:new()
if not db then
ngx.say("failed to instantiate mysql: ", err)
return
end
db:set_timeout(1000) -- 1 sec
-- or connect to a unix domain socket file listened
-- by a mysql server:
-- local ok, err, errcode, sqlstate =
-- db:connect{
-- path = "/path/to/mysql.sock",
-- database = "ngx_test",
-- user = "ngx_test",
-- password = "ngx_test" }
local ok, err, errcode, sqlstate = db:connect{
-- host = "mysql.nginx-ingress.svc.cluster.local",
host = "mysql.nginx-ingress.svc.cluster.local",
port = 3306,
database = "my_database",
user = "root",
password = "2ZjMDPcySI",
charset = "utf8",
max_packet_size = 1024 * 1024,
}
if not ok then
ngx.say("failed to connect: ", err, ": ", errcode, " ", sqlstate)
return
end
ngx.say("connected to mysql.")
local res, err, errcode, sqlstate =
db:query("drop table if exists cats")
if not res then
ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
return
end
res, err, errcode, sqlstate =
db:query("create table cats "
.. "(id serial primary key, "
.. "name varchar(5))")
if not res then
ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
return
end
ngx.say("table cats created.")
res, err, errcode, sqlstate =
db:query("insert into cats (name) "
.. "values ('Bob'),(''),(null)")
if not res then
ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
return
end
ngx.say(res.affected_rows, " rows inserted into table cats ",
"(last insert id: ", res.insert_id, ")")
-- run a select query, expected about 10 rows in
-- the result set:
res, err, errcode, sqlstate =
db:query("select * from cats order by id asc", 10)
if not res then
ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
return
end
local cjson = require "cjson"
ngx.say("result: ", cjson.encode(res))
-- put it into the connection pool of size 100,
-- with 10 seconds max idle timeout
local ok, err = db:set_keepalive(10000, 100)
if not ok then
ngx.say("failed to set keepalive: ", err)
return
end
-- or just close the connection right away:
-- local ok, err = db:close()
-- if not ok then
-- ngx.say("failed to close: ", err)
-- return
-- end
代码语言:javascript复制local cjson = require "cjson"
if ngx.var.my_headers ~= nil
then
headers = cjson.decode(ngx.var.my_headers)
for k, v in pairs(headers) do
ngx.header[k] = v
end
end
ngx.header["Content-Type"]="text/plain"
测试一下:
代码语言:javascript复制% curl http://localhost:31307/lua
hello lua file test
10020
connected to mysql.
table cats created.
3 rows inserted into table cats (last insert id: 1)
result: [{"id":"1","name":"Bob"},{"id":"2","name":""},{"id":"3","name":null}]
可以看到我们已经成功从mysql取到内容返回回来了