mac 上学习k8s系列(45)openresty操作mysql

2022-08-02 19:43:38 浏览数 (1)

Nginx Ingress 由资源对象 Ingress、Ingress 控制器、Nginx 三部分组成,Ingress 控制器用以将 Ingress 资源实例组装成 Nginx 配置文件(nginx.conf),并重新加载 Nginx 使变更的配置生效。当它监听到 Service 中 Pod 变化时通过动态变更的方式实现 Nginx 上游服务器组配置的变更,无须重新加载 Nginx 进程。

修改nginx.conf有三种方式,1,直接修改nginx.conf,2,configMap;3,Ingress;其中configMap可以作为基础模板,它能改动的地方更多,分为下面4个部分

  • main-snippet string "" 在 main 指令域添加 Nginx 配置指令
  • http-snippet string "" 在 http 指令域添加 Nginx 配置指令
  • server-snippet string "" 在 server 指令域添加 Nginx 配置指令
  • location-snippet string "" 在 location 指令域添加 Nginx 配置指令

而Ingress只能修改server和location域的指令,具体是通过annotations:

  • nginx.ingress.kubernetes.io/server-snippet string 在 server 指令域添加 Nginx 配置指令
  • nginx.ingress.kubernetes.io/configuration-snippet string 在 location 指令域添加 Nginx 配置指令

首选我们搭建openresty环境,通过直接修改nginx.conf的方式来用lua连接mysql

代码语言:javascript复制
helm install ingress-nginx bitnami/nginx-ingress-controller
NAME: ingress-nginx
LAST DEPLOYED: Fri Jun  3 22:29:03 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1

把镜像更新一下

代码语言:javascript复制
% kubectl -n nginx-ingress edit deploy nginx-ingress-nginx-ingress-controller
 image: docker.io/bitnami/nginx-ingress-controller:latest
 
 % kubectl -n nginx-ingress edit deployment.apps/nginx-ingress-nginx-ingress-controller-default-backend
 image: docker.io/bitnami/openresty:latest

修改nginx连接方式为nodePort

代码语言:javascript复制
 % kubectl -n nginx-ingress edit svc nginx-ingress-nginx-ingress-controller-default-backend
service/nginx-ingress-nginx-ingress-controller-default-backend edited
 type: NodePort

测试下

代码语言:javascript复制
 % curl http://127.0.0.1:31307/

openresty已经起来了,但是由于配置文件在pod里面,因此我们需要挂载一个volume,把配置文件映射到主机上

代码语言:javascript复制
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-volume
spec:
  storageClassName: hostpath
  capacity:
    storage: 200Mi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/Users/xiazemin/go/src/github.com/xiazemin/k8s_learn/ingress/lua/"
代码语言:javascript复制
% kubectl apply -f ingress/lua/pv.yaml
persistentvolume/pv-volume created
代码语言:javascript复制
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-volume
spec:
  storageClassName: hostpath
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Mi
代码语言:javascript复制
 % kubectl apply -f ingress/lua/pvc.yaml
persistentvolumeclaim/pvc-volume created

在部署中映射主机目录

代码语言:javascript复制
       volumeMounts:
        - name: pv-volume
          mountPath: /opt/bitnami/openresty/nginx/conf/server_blocks/
      volumes:
      - name: pv-volume
        persistentVolumeClaim:
          claimName: pvc-volume

% kubectl -n nginx-ingress edit deployment.apps/nginx-ingress-nginx-ingress-controller-default-backend

注意容器内的路径使用

代码语言:javascript复制
/opt/bitnami/openresty/nginx/conf/server_blocks/

这个bitnami的helm chart上抽象好的nginx配置文件加载路径,否则需要更改docker镜像,设置环境变量,巨麻烦。

修改nginx.conf文件

代码语言:javascript复制
# Based on https://www.nginx.com/resources/wiki/start/topics/examples/full/#nginx-conf

# you do not need the following line if you are using
    # the ngx_openresty bundle:
server {
    # Port to listen on, can also be set in IP:PORT format
    listen  8080;
    location / {
        stub_status on;
        access_log   on;
        content_by_lua_block {
            ngx.say("hello lua");
        }
    }
}

重启pod,测试下

代码语言:javascript复制
 % curl http://localhost:31307/
hello lua

说明基础环境已经成功了,下面我们连接mysql,首先部署mysql

代码语言:javascript复制
% helm install mysql bitnami/mysql -n nginx-ingress
NAME: mysql
LAST DEPLOYED: Sun Jun  5 13:31:57 2022
NAMESPACE: nginx-ingress
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
** Please be patient while the chart is being deployed **

Tip:

修改nginx.conf支持加载lua文件

代码语言:javascript复制
# Based on https://www.nginx.com/resources/wiki/start/topics/examples/full/#nginx-conf

# you do not need the following line if you are using
    # the ngx_openresty bundle:
lua_package_path "/opt/bitnami/openresty/nginx/conf/server_blocks/rocks-5.1/lua-resty-mysql/lib/?.lua;;";

server {
    lua_code_cache off;
    # Port to listen on, can also be set in IP:PORT format
    listen  8080;
    location /lua {
        resolver kube-dns.kube-system.svc.cluster.local;
        access_log on;
        content_by_lua_file /opt/bitnami/openresty/nginx/conf/server_blocks/hello.lua;
        header_filter_by_lua_file /opt/bitnami/openresty/nginx/conf/server_blocks/header.lua;
    }
    location / {
        resolver kube-dns.kube-system.svc.cluster.local;
        stub_status on;
        access_log   on;
        content_by_lua_block {
            ngx.say("hello lua");
        }
    }
}

关闭lua的代码缓冲,可以方便热更新,不用重启nginx

代码语言:javascript复制
 lua_code_cache off;

指定lua包的路径

代码语言:javascript复制
lua_package_path "/opt/bitnami/openresty/nginx/conf/server_blocks/rocks-5.1/lua-resty-mysql/lib/?.lua;;";

lua-resty-mysql 是通过包管理工具luarocks下载的

代码语言:javascript复制
 % luarocks --lua-dir=/usr/local/opt/lua@5.1 install lua-resty-mysql 0.15-0

安装成功后把路径链接到我们映射的目录里面,我们在代码里面就可以直接引用了。

注意nginx需要在location里指定域名解析器,否则没法访问我们内部的service

代码语言:javascript复制
 resolver kube-dns.kube-system.svc.cluster.local;

下面是我们修改内容和header的两段代码

代码语言:javascript复制
ngx.say("hello lua file test");
ngx.say(ngx.config.ngx_lua_version);


local mysql = require "resty.mysql"
local db, err = mysql:new()
if not db then
    ngx.say("failed to instantiate mysql: ", err)
    return
end

db:set_timeout(1000) -- 1 sec

-- or connect to a unix domain socket file listened
-- by a mysql server:
--     local ok, err, errcode, sqlstate =
--           db:connect{
--              path = "/path/to/mysql.sock",
--              database = "ngx_test",
--              user = "ngx_test",
--              password = "ngx_test" }

local ok, err, errcode, sqlstate = db:connect{
    -- host = "mysql.nginx-ingress.svc.cluster.local",
    host = "mysql.nginx-ingress.svc.cluster.local",
    port = 3306,
    database = "my_database",
    user = "root",
    password = "2ZjMDPcySI",
    charset = "utf8",
    max_packet_size = 1024 * 1024,
}

if not ok then
    ngx.say("failed to connect: ", err, ": ", errcode, " ", sqlstate)
    return
end

ngx.say("connected to mysql.")

local res, err, errcode, sqlstate =
    db:query("drop table if exists cats")
if not res then
    ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
    return
end

res, err, errcode, sqlstate =
    db:query("create table cats "
                .. "(id serial primary key, "
                .. "name varchar(5))")
if not res then
    ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
    return
end

ngx.say("table cats created.")

res, err, errcode, sqlstate =
    db:query("insert into cats (name) "
                .. "values ('Bob'),(''),(null)")
if not res then
    ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
    return
end

ngx.say(res.affected_rows, " rows inserted into table cats ",
        "(last insert id: ", res.insert_id, ")")

-- run a select query, expected about 10 rows in
-- the result set:
res, err, errcode, sqlstate =
    db:query("select * from cats order by id asc", 10)
if not res then
    ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
    return
end

local cjson = require "cjson"
ngx.say("result: ", cjson.encode(res))

-- put it into the connection pool of size 100,
-- with 10 seconds max idle timeout
local ok, err = db:set_keepalive(10000, 100)
if not ok then
    ngx.say("failed to set keepalive: ", err)
    return
end

-- or just close the connection right away:
-- local ok, err = db:close()
-- if not ok then
--     ngx.say("failed to close: ", err)
--     return
-- end
代码语言:javascript复制
local cjson = require "cjson"

if ngx.var.my_headers ~= nil
then
    headers = cjson.decode(ngx.var.my_headers)
    for k, v in pairs(headers) do
        ngx.header[k] = v
    end
end

ngx.header["Content-Type"]="text/plain"
            

测试一下:

代码语言:javascript复制
% curl  http://localhost:31307/lua
hello lua file test
10020
connected to mysql.
table cats created.
3 rows inserted into table cats (last insert id: 1)
result: [{"id":"1","name":"Bob"},{"id":"2","name":""},{"id":"3","name":null}]

可以看到我们已经成功从mysql取到内容返回回来了

0 人点赞