Kubernetes (K8S) 中安装部署APISIX
王先森2023-09-252023-09-25
APISIX
Apache APISIX 是一个基于 OpenResty 和 Etcd 实现的动态、实时、高性能、可扩展的微服务 API 网关，目前已经是 Apache 顶级项目。提供了丰富的流量管理功能，如负载均衡、动态路由、动态 upstream、A/B 测试、金丝雀发布、限速、熔断、防御恶意攻击、认证、监控指标、服务可观测性、服务治理等。可以使用 APISIX 来处理传统的南北流量以及服务之间的东西向流量。
APISIX 基于 Nginx 和 etcd，与传统 API 网关相比，APISIX 具有动态路由和热加载插件功能，避免了配置之后的 reload 操作，同时 APISIX 支持 HTTP(S)、HTTP2、Dubbo、QUIC、MQTT、TCP/UDP 等更多的协议。而且还内置了 Dashboard，提供强大而灵活的界面。同样也提供了丰富的插件支持功能，而且还可以让用户自定义插件。
上图是 APISIX 的架构图，整体上分成数据面和控制面两个部分，控制面用来管理路由，主要通过 etcd 来实现配置中心，数据面用来处理客户端请求，通过 APISIX 自身来实现，会不断去 watch etcd 中的 route、upstream 等数据。
APISIX Ingress
同样作为一个 API 网关，APISIX 也支持作为 Kubernetes 的一个 Ingress 控制器进行使用。APISIX Ingress 在架构上分成了两部分，一部分是 APISIX Ingress Controller，作为控制面它将完成配置管理与分发。另一部分 APISIX(代理) 负责承载业务流量。
当 Client 发起请求，到达 Apache APISIX 后，会直接把相应的业务流量传输到后端（如 Service Pod），从而完成转发过程。此过程不需要经过 Ingress Controller，这样做可以保证一旦有问题出现，或者是进行变更、扩缩容或者迁移处理等，都不会影响到用户和业务流量。
同时在配置端，用户通过 kubectl apply 创建资源，可将自定义 CRD 配置应用到 K8s 集群，Ingress Controller 会持续 watch 这些资源变更，来将相应配置应用到 Apache APISIX（通过 admin api）。
从上图可以看出 APISIX Ingress 采用了数据面与控制面的分离架构，所以用户可以选择将数据面部署在 K8s 集群内部或外部。但 Ingress Nginx 是将控制面和数据面放在了同一个 Pod 中，如果 Pod 或控制面出现一点闪失，整个 Pod 就会挂掉，进而影响到业务流量。这种架构分离，给用户提供了比较方便的部署选择，同时在业务架构调整场景下，也方便进行相关数据的迁移与使用。
APISIX Ingress 控制器目前支持的核心特性包括：
- 全动态，支持高级路由匹配规则，可与 Apache APISIX 官方 50 多个插件 & 客户自定义插件进行扩展使用
- 支持 CRD，更容易理解声明式配置
- 兼容原生 Ingress 资源对象
- 支持流量切分
- 服务自动注册发现，无惧扩缩容
- 更灵活的负载均衡策略，自带健康检查功能
- 支持 gRPC plaintext 与 TCP 4 层代理
APISIX安装
Helm安装
我们这里在 Kubernetes 集群中来使用 APISIX，可以通过 Helm Chart 来进行安装，首先添加官方提供的 Helm Chart 仓库：
代码语言：javascript复制helm repo add apisix https://charts.apiseven.com
helm repo update
helm install apisix apisix/apisix --create-namespace --namespace apisix
# 启动一些其他配置
helm repo add apisix https://charts.apiseven.com && helm repo update && helm upgrade --install apisix apisix/apisix --create-namespace --namespace apisix --set dashboard.enabled=true --set ingress-controller.enabled=true --set ingress-controller.config.apisix.serviceNamespace=apisixAPISIX参考文档：https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix/README.md
APISIX-Ingress参考文档：https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix-ingress-controller/README.md
APISIX-Dashboard参考文档：https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix-dashboard/README.md
资源配置清单安装
PAISIX
准备资源配置清单
ConfigMapDeploymentService
代码语言：javascript复制vim cm.yml
apiVersion: v1
kind: ConfigMap
metadata:
name: apisix
namespace: apisix
data:
config.yaml: |-
apisix:
# node_listen: 9080 # APISIX listening port.
node_listen: # APISIX listening ports.
- 9080
# - port: 9081
# enable_http2: true # If not set, default to `false`.
# - ip: 127.0.0.2 # If not set, default to `0.0.0.0`
# port: 9082
# enable_http2: true
enable_admin: true # Admin API
enable_dev_mode: false # If true, set nginx `worker_processes` to 1.
enable_reuseport: true # If true, enable nginx SO_REUSEPORT option.
show_upstream_status_in_response_header: false # If true, include the upstream HTTP status code in
# the response header `X-APISIX-Upstream-Status`.
# If false, show `X-APISIX-Upstream-Status` only if
# the upstream response code is 5xx.
enable_ipv6: true
enable_server_tokens: true # If true, show APISIX version in the `Server` response header.
extra_lua_path: "" # Extend lua_package_path to load third-party code.
extra_lua_cpath: "" # Extend lua_package_cpath to load third-party code.
# lua_module_hook: "my_project.my_hook" # Hook module used to inject third-party code into APISIX.
proxy_cache: # Proxy Caching configuration
cache_ttl: 10s # The default caching time on disk if the upstream does not specify a caching time.
zones:
- name: disk_cache_one # Name of the cache.
memory_size: 50m # Size of the memory to store the cache index.
disk_size: 1G # Size of the disk to store the cache data.
disk_path: /tmp/disk_cache_one # Path to the cache file for disk cache.
cache_levels: 1:2 # Cache hierarchy levels of disk cache.
# - name: disk_cache_two
# memory_size: 50m
# disk_size: 1G
# disk_path: "/tmp/disk_cache_two"
# cache_levels: "1:2"
- name: memory_cache
memory_size: 50m
delete_uri_tail_slash: false # Delete the '/' at the end of the URI
normalize_uri_like_servlet: false # If true, use the same path normalization rules as the Java
# servlet specification. See https://github.com/jakartaee/servlet/blob/master/spec/src/main/asciidoc/servlet-spec-body.adoc#352-uri-path-canonicalization, which is used in Tomcat.
router:
http: radixtree_host_uri # radixtree_host_uri: match route by host and URI
# radixtree_uri: match route by URI
# radixtree_uri_with_parameter: similar to radixtree_uri but match URI with parameters. See https://github.com/api7/lua-resty-radixtree/#parameters-in-path for more details.
ssl: radixtree_sni # radixtree_sni: match route by SNI
# http is the default proxy mode. proxy_mode can be one of `http`, `stream`, or `http&stream`
proxy_mode: http
# stream_proxy: # TCP/UDP L4 proxy
# only: true # Enable L4 proxy only without L7 proxy.
# tcp:
# - addr: 9100 # Set the TCP proxy listening ports.
# tls: true
# - addr: "127.0.0.1:9101"
# udp: # Set the UDP proxy listening ports.
# - 9200
# - "127.0.0.1:9201"
# dns_resolver: # If not set, read from `/etc/resolv.conf`
# - 1.1.1.1
# - 8.8.8.8
# dns_resolver_valid: 30 # Override the default TTL of the DNS records.
resolver_timeout: 5 # Set the time in seconds that the server will wait for a response from the
# DNS resolver before timing out.
enable_resolv_search_opt: true # If true, use search option in the resolv.conf file in DNS lookups.
ssl:
enable: true
listen: # APISIX listening port for HTTPS traffic.
- port: 9443
enable_http2: true
# - ip: 127.0.0.3 # If not set, default to `0.0.0.0`.
# port: 9445
# enable_http2: true
ssl_trusted_certificate: /usr/local/apisix/ssl/ca.pem # 此处需要修改成连接etcd的ca证书位置，开启https连接必须配置
ssl_protocols: TLSv1.2 TLSv1.3 # TLS versions supported.
ssl_ciphers: ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl_session_tickets: false # If true, session tickets are used for SSL/TLS connections.
# Disabled by default because it renders Perfect Forward Secrecy (FPS)
# useless. See https://github.com/mozilla/server-side-tls/issues/135.
key_encrypt_salt: # This field is only used to encrypt the private key of SSL.
- edd1c9f0985e76a2 # Set the encryption key for AES-128-CBC. It should be a
# hexadecimal string of length 16.
# If not set, APISIX saves the original data into etcd.
# CAUTION: If you would like to update the key, add the new key as the
# first item in the array and keep the older keys below the newly added
# key, so that data can be decrypted with the older keys and encrypted
# with the new key. Removing the old keys directly can render the data
# unrecoverable.
# fallback_sni: "my.default.domain" # Fallback SNI to be used if the client does not send SNI during
# # the handshake.
enable_control: true # Control API
# control:
# ip: 127.0.0.1
# port: 9090
disable_sync_configuration_during_start: false # Safe exit. TO BE REMOVED.
data_encryption: # Encrypt fields specified in `encrypt_fields` in plugin schema.
enable: false
keyring: # Set the encryption key for AES-128-CBC. It should be a
- qeddd145sfvddff3 # hexadecimal string of length 16.
# If not set, APISIX saves the original data into etcd.
# CAUTION: If you would like to update the key, add the new key as the
# first item in the array and keep the older keys below the newly added
# key, so that data can be decrypted with the older keys and encrypted
# with the new key. Removing the old keys directly can render the data
# unrecoverable.
nginx_config: # Config for render the template to generate nginx.conf
# user: root # Set the execution user of the worker process. This is only
# effective if the master process runs with super-user privileges.
error_log: logs/error.log # Location of the error log.
error_log_level: warn # Logging level: info, debug, notice, warn, error, crit, alert, or emerg.
worker_processes: auto # Automatically determine the optimal number of worker processes based
# on the available system resources.
# If you want use multiple cores in container, you can inject the number of
# CPU cores as environment variable "APISIX_WORKER_PROCESSES".
enable_cpu_affinity: false # Disable CPU affinity by default as worker_cpu_affinity affects the
# behavior of APISIX in containers. For example, multiple instances could
# be bound to one CPU core, which is not desirable.
# If APISIX is deployed on a physical machine, CPU affinity can be enabled.
worker_rlimit_nofile: 20480 # The number of files a worker process can open.
# The value should be larger than worker_connections.
worker_shutdown_timeout: 240s # Timeout for a graceful shutdown of worker processes.
max_pending_timers: 16384 # The maximum number of pending timers that can be active at any given time.
# Error "too many pending timers" indicates the threshold is reached.
max_running_timers: 4096 # The maximum number of running timers that can be active at any given time.
# Error "lua_max_running_timers are not enough" error indicates the
# threshold is reached.
event:
worker_connections: 10620
# envs: # Get environment variables.
# - TEST_ENV
meta:
lua_shared_dict: # Nginx Lua shared memory zone. Size units are m or k.
prometheus-metrics: 15m
stream:
enable_access_log: false # Enable stream proxy access logging.
access_log: logs/access_stream.log # Location of the stream access log.
access_log_format: "$remote_addr [$time_local] $protocol $status $bytes_sent $bytes_received $session_time" # Customize log format: http://nginx.org/en/docs/varindex.html
access_log_format_escape: default # Escape default or json characters in variables.
lua_shared_dict: # Nginx Lua shared memory zone. Size units are m or k.
etcd-cluster-health-check-stream: 10m
lrucache-lock-stream: 10m
plugin-limit-conn-stream: 10m
worker-events-stream: 10m
tars-stream: 1m
# Add other custom Nginx configurations.
# Users are responsible for validating the custom configurations
# to ensure they are not in conflict with APISIX configurations.
main_configuration_snippet: |
# Add custom Nginx main configuration to nginx.conf.
# The configuration should be well indented!
http_configuration_snippet: |
# Add custom Nginx http configuration to nginx.conf.
# The configuration should be well indented!
http_server_configuration_snippet: |
# Add custom Nginx http server configuration to nginx.conf.
# The configuration should be well indented!
http_server_location_configuration_snippet: |
# Add custom Nginx http server location configuration to nginx.conf.
# The configuration should be well indented!
http_admin_configuration_snippet: |
# Add custom Nginx admin server configuration to nginx.conf.
# The configuration should be well indented!
http_end_configuration_snippet: |
# Add custom Nginx http end configuration to nginx.conf.
# The configuration should be well indented!
stream_configuration_snippet: |
# Add custom Nginx stream configuration to nginx.conf.
# The configuration should be well indented!
http:
enable_access_log: true # Enable HTTP proxy access logging.
access_log: logs/access.log # Location of the access log.
access_log_buffer: 16384 # buffer size of access log.
access_log_format: "$remote_addr - $remote_user [$time_local] $http_host "$request" $status $body_bytes_sent $request_time "$http_referer" "$http_user_agent" $upstream_addr $upstream_status $upstream_response_time "$upstream_scheme://$upstream_host$upstream_uri""
# Customize log format: http://nginx.org/en/docs/varindex.html
access_log_format_escape: default # Escape default or json characters in variables.
keepalive_timeout: 60s # Set the maximum time for which TCP connection keeps alive.
client_header_timeout: 60s # Set the maximum time waiting for client to send the entire HTTP
# request header before closing the connection.
client_body_timeout: 60s # Set the maximum time waiting for client to send the request body.
client_max_body_size: 0 # Set the maximum allowed size of the client request body.
# Default to 0, unlimited.
# Unlike Nginx, APISIX does not limit the body size by default.
# If exceeded, the 413 (Request Entity Too Large) error is returned.
send_timeout: 10s # Set the maximum time for transmitting a response to the client before closing.
underscores_in_headers: "on" # Allow HTTP request headers to contain underscores in their names.
real_ip_header: X-Real-IP # https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header
real_ip_recursive: "off" # http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_recursive
real_ip_from: # http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from
- 127.0.0.1
- "unix:"
# custom_lua_shared_dict: # Custom Nginx Lua shared memory zone for nginx.conf. Size units are m or k.
# ipc_shared_dict: 100m # Custom shared cache, format: `cache-key: cache-size`
proxy_ssl_server_name: true # Send the server name in the SNI extension when establishing an SSL/TLS
# connection with the upstream server, allowing the upstream server to
# select the appropriate SSL/TLS certificate and configuration based on
# the requested server name.
upstream:
keepalive: 320 # Set the maximum time of keep-alive connections to the upstream servers.
# When the value is exceeded, the least recently used connection is closed.
keepalive_requests: 1000 # Set the maximum number of requests that can be served through one
# keep-alive connection.
# After the maximum number of requests is made, the connection is closed.
keepalive_timeout: 60s # Set the maximum time for which TCP connection keeps alive.
charset: utf-8 # Add the charset to the "Content-Type" response header field.
# See http://nginx.org/en/docs/http/ngx_http_charset_module.html#charset
variables_hash_max_size: 2048 # Set the maximum size of the variables hash table.
lua_shared_dict: # Nginx Lua shared memory zone. Size units are m or k.
internal-status: 10m
plugin-limit-req: 10m
plugin-limit-count: 10m
prometheus-metrics: 10m
plugin-limit-conn: 10m
upstream-healthcheck: 10m
worker-events: 10m
lrucache-lock: 10m
balancer-ewma: 10m
balancer-ewma-locks: 10m
balancer-ewma-last-touched-at: 10m
plugin-limit-count-redis-cluster-slot-lock: 1m
tracing_buffer: 10m
plugin-api-breaker: 10m
etcd-cluster-health-check: 10m
discovery: 1m
jwks: 1m
introspection: 10m
access-tokens: 1m
ext-plugin: 1m
tars: 1m
cas-auth: 10m
# discovery: # Service Discovery
# dns:
# servers:
# - "127.0.0.1:8600" # Replace with the address of your DNS server.
# order: # Resolve DNS records this order.
# - last # Try the latest successful type for a hostname.
# - SRV
# - A
# - AAAA
# - CNAME
graphql:
max_size: 1048576 # Set the maximum size limitation of graphql in bytes. Default to 1MiB.
plugins: # plugin list (sorted by priority)
- real-ip # priority: 23000
- ai # priority: 22900
- client-control # priority: 22000
- proxy-control # priority: 21990
- request-id # priority: 12015
- zipkin # priority: 12011
#- skywalking # priority: 12010
#- opentelemetry # priority: 12009
- ext-plugin-pre-req # priority: 12000
- fault-injection # priority: 11000
- mocking # priority: 10900
- serverless-pre-function # priority: 10000
#- batch-requests # priority: 4010
- cors # priority: 4000
- ip-restriction # priority: 3000
- ua-restriction # priority: 2999
- referer-restriction # priority: 2990
- csrf # priority: 2980
- uri-blocker # priority: 2900
- request-validation # priority: 2800
- chaitin-waf # priority: 2700
- openid-connect # priority: 2599
- cas-auth # priority: 2597
- authz-casbin # priority: 2560
- authz-casdoor # priority: 2559
- wolf-rbac # priority: 2555
- ldap-auth # priority: 2540
- hmac-auth # priority: 2530
- basic-auth # priority: 2520
- jwt-auth # priority: 2510
- key-auth # priority: 2500
- consumer-restriction # priority: 2400
- forward-auth # priority: 2002
- opa # priority: 2001
- authz-keycloak # priority: 2000
#- error-log-logger # priority: 1091
- proxy-cache # priority: 1085
- body-transformer # priority: 1080
- proxy-mirror # priority: 1010
- proxy-rewrite # priority: 1008
- workflow # priority: 1006
- api-breaker # priority: 1005
- limit-conn # priority: 1003
- limit-count # priority: 1002
- limit-req # priority: 1001
#- node-status # priority: 1000
- gzip # priority: 995
- server-info # priority: 990
- traffic-split # priority: 966
- redirect # priority: 900
- response-rewrite # priority: 899
- degraphql # priority: 509
- kafka-proxy # priority: 508
#- dubbo-proxy # priority: 507
- grpc-transcode # priority: 506
- grpc-web # priority: 505
- public-api # priority: 501
- prometheus # priority: 500
- datadog # priority: 495
- loki-logger # priority: 414
- elasticsearch-logger # priority: 413
- echo # priority: 412
- loggly # priority: 411
- http-logger # priority: 410
- splunk-hec-logging # priority: 409
- skywalking-logger # priority: 408
- google-cloud-logging # priority: 407
- sls-logger # priority: 406
- tcp-logger # priority: 405
- kafka-logger # priority: 403
- rocketmq-logger # priority: 402
- syslog # priority: 401
- udp-logger # priority: 400
- file-logger # priority: 399
- clickhouse-logger # priority: 398
- tencent-cloud-cls # priority: 397
- inspect # priority: 200
#- log-rotate # priority: 100
# <- recommend to use priority (0, 100) for your custom plugins
- example-plugin # priority: 0
#- gm # priority: -43
- aws-lambda # priority: -1899
- azure-functions # priority: -1900
- openwhisk # priority: -1901
- openfunction # priority: -1902
- serverless-post-function # priority: -2000
- ext-plugin-post-req # priority: -3000
- ext-plugin-post-resp # priority: -4000
stream_plugins: # stream plugin list (sorted by priority)
- ip-restriction # priority: 3000
- limit-conn # priority: 1003
- mqtt-proxy # priority: 1000
#- prometheus # priority: 500
- syslog # priority: 401
# <- recommend to use priority (0, 100) for your custom plugins
plugin_attr: # Plugin attributes
log-rotate: # Plugin: log-rotate
interval: 3600 # Set the log rotate interval in seconds.
max_kept: 168 # Set the maximum number of log files to keep. If exceeded, historic logs are deleted.
max_size: -1 # Set the maximum size of log files in bytes before a rotation.
# Skip size check if max_size is less than 0.
enable_compression: false # Enable log file compression (gzip).
skywalking: # Plugin: skywalking
service_name: APISIX # Set the service name for SkyWalking reporter.
service_instance_name: APISIX Instance Name # Set the service instance name for SkyWalking reporter.
endpoint_addr: http://127.0.0.1:12800 # Set the SkyWalking HTTP endpoint.
report_interval: 3 # Set the reporting interval in second.
opentelemetry: # Plugin: opentelemetry
trace_id_source: x-request-id # Specify the source of the trace ID for OpenTelemetry traces.
resource:
service.name: APISIX # Set the service name for OpenTelemetry traces.
collector:
address: 127.0.0.1:4318 # Set the address of the OpenTelemetry collector to send traces to.
request_timeout: 3 # Set the timeout for requests to the OpenTelemetry collector in seconds.
request_headers: # Set the headers to include in requests to the OpenTelemetry collector.
Authorization: token # Set the authorization header to include an access token.
batch_span_processor:
drop_on_queue_full: false # Drop spans when the export queue is full.
max_queue_size: 1024 # Set the maximum size of the span export queue.
batch_timeout: 2 # Set the timeout for span batches to wait in the export queue before
# being sent.
inactive_timeout: 1 # Set the timeout for spans to wait in the export queue before being sent,
# if the queue is not full.
max_export_batch_size: 16 # Set the maximum number of spans to include in each batch sent to the
set_ngx_var: false # export opentelemetry variables to nginx variables
# OpenTelemetry collector.
prometheus: # Plugin: prometheus
export_uri: /apisix/prometheus/metrics # Set the URI for the Prometheus metrics endpoint.
metric_prefix: apisix_ # Set the prefix for Prometheus metrics generated by APISIX.
enable_export_server: true # Enable the Prometheus export server.
export_addr: # Set the address for the Prometheus export server.
ip: 127.0.0.1 # Set the IP.
port: 9091 # Set the port.
# metrics: # Create extra labels from nginx variables: https://nginx.org/en/docs/varindex.html
# http_status:
# extra_labels:
# - upstream_addr: $upstream_addr
# - status: $upstream_status # The label name does not need to be the same as the variable name.
# http_latency:
# extra_labels:
# - upstream_addr: $upstream_addr
# bandwidth:
# extra_labels:
# - upstream_addr: $upstream_addr
# default_buckets:
# - 10
# - 50
# - 100
# - 200
# - 500
server-info: # Plugin: server-info
report_ttl: 60 # Set the TTL in seconds for server info in etcd.
# Maximum: 86400. Minimum: 3.
dubbo-proxy: # Plugin: dubbo-proxy
upstream_multiplex_count: 32 # Set the maximum number of connections that can be multiplexed over
# a single network connection between the Dubbo Proxy and the upstream
# Dubbo services.
proxy-mirror: # Plugin: proxy-mirror
timeout: # Set the timeout for mirrored requests.
connect: 60s
read: 60s
send: 60s
# redirect: # Plugin: redirect
# https_port: 8443 # Set the default port used to redirect HTTP to HTTPS.
inspect: # Plugin: inspect
delay: 3 # Set the delay in seconds for the frequency of checking the hooks file.
hooks_file: "/usr/local/apisix/plugin_inspect_hooks.lua" # Set the path to the Lua file that defines
# hooks. Only administrators should have
# write access to this file for security.
deployment: # Deployment configurations
role: traditional # Set deployment mode: traditional, control_plane, or data_plane.
role_traditional:
config_provider: etcd # Set the configuration center.
#role_data_plane: # Set data plane details if role is data_plane.
# config_provider: etcd # Set the configuration center: etcd, xds, or yaml.
#role_control_plane: # Set control plane details if role is control_plane.
# config_provider: etcd # Set the configuration center.
admin: # Admin API
admin_key_required: true # Enable Admin API authentication by default for security.
admin_key:
-
name: admin # admin: write access to configurations.
key: edd1c9f034335f136f87ad84b625c8f1 # Set API key for the admin of Admin API.
role: admin
-
name: viewer # viewer: read-only to configurations.
key: 4054f7cf07e344346cd3f287985e76a2 # Set API key for the viewer of Admin API.
role: viewer
enable_admin_cors: true # Enable Admin API CORS response header `Access-Control-Allow-Origin`.
allow_admin: # Limit Admin API access by IP addresses.
- 127.0.0.0/24 # If not set, any IP address is allowed.
- 0.0.0.0/0
# - "::/64"
admin_listen: # Set the Admin API listening addresses.
ip: 0.0.0.0 # Set listening IP.
port: 9180 # Set listening port. Beware of port conflict with node_listen.
# https_admin: true # Enable SSL for Admin API on IP and port specified in admin_listen.
# Use admin_api_mtls.admin_ssl_cert and admin_api_mtls.admin_ssl_cert_key.
# admin_api_mtls: # Set this if `https_admin` is true.
# admin_ssl_cert: "" # Set path to SSL/TLS certificate.
# admin_ssl_cert_key: "" # Set path to SSL/TLS key.
# admin_ssl_ca_cert: "" # Set path to CA certificate used to sign client certificates.
admin_api_version: v3 # Set the version of Admin API (latest: v3).
etcd:
host:
# - "https://10.1.1.100:2379"
# - "https://10.1.1.120:2379"
# - "https://10.1.1.130:2379"
- "https://etcd1.local.host:2379"
- "https://etcd2.local.host:2379"
- "https://etcd3.local.host:2379"
prefix: "/apisix" # apisix configurations prefix
timeout: 30 # seconds
tls:
verify: true
cert: "/etcd-ssl/tls.crt"
key: "/etcd-ssl/tls.key"
sni: "*.local.host"代码语言：javascript复制vim dp.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: apisix
app.kubernetes.io/version: 3.5.0
name: apisix
namespace: apisix
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: apisix
template:
metadata:
labels:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: apisix
spec:
nodeSelector: # 这里选择标签为app=apisix节点。
app: apisix
containers:
- image: apache/apisix:3.5.0-debian
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /bin/sh
- -c
- sleep 30
name: apisix
ports:
- containerPort: 9080
name: http
protocol: TCP
- containerPort: 9443
name: tls
protocol: TCP
- containerPort: 9180
name: admin
protocol: TCP
readinessProbe:
failureThreshold: 6
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 9080
timeoutSeconds: 1
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /usr/local/apisix/conf/config.yaml
name: apisix-config
subPath: config.yaml
- mountPath: /etcd-ssl/ # etcd证书目录，与configMap一致。
name: etcd-tls
- mountPath: /usr/local/apisix/ssl/ca.pem # ca证书路径，与configMap一致。
name: apisix-ca
subPath: ca.pem
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
name: apisix
name: apisix-config
- secret:
secretName: etcd-tls # 存放etcd证书的Secret
name: etcd-tls
- secret:
secretName: apisix-ca # 存放ca证书的Secret
name: apisix-ca代码语言：javascript复制vim svc.yml
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: apisix
app.kubernetes.io/service: apisix-admin
app.kubernetes.io/version: 3.5.0
name: apisix-admin
namespace: apisix
spec:
ports:
- name: apisix-admin
port: 9180
protocol: TCP
targetPort: 9180
selector:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: apisix
sessionAffinity: None
type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: apisix
app.kubernetes.io/service: apisix-gateway
app.kubernetes.io/version: 3.5.0
name: apisix-gateway
namespace: apisix
spec:
externalTrafficPolicy: Cluster
ipFamilyPolicy: SingleStack
ports:
- name: apisix-gateway-http
nodePort: 30080
port: 80
protocol: TCP
targetPort: 9080
- name: apisix-gateway-https
nodePort: 30443
port: 443
protocol: TCP
targetPort: 9443
selector:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: apisix
sessionAffinity: None
type: NodePort应用资源配置清单
代码语言：javascript复制# 创建名称空间
kubectl create ns apisix
# 创建所需要的secret
# 注意 etcd证书需要带有域名信息
kubectl create secret tls etcd-tls --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem -n apisix
kubectl create secret generic apisix-ca --from-file=/opt/etcd/ssl/ca.pem -n apisix
# 创建
kubectl apply -f cm.yml
kubectl apply -f dp.yml
kubectl apply -f svc.ymlAPISIX-Ingress
准备资源配置清单
ConfigMapDeploymentRBACCRDService
代码语言：javascript复制vim cm.yml
apiVersion: v1
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: ingress-controller
app.kubernetes.io/version: 1.6.0
name: apisix-configmap
namespace: apisix
data:
config.yaml: |-
# log options
log_level: "info"
log_output: "stderr"
cert_file: "/etc/webhook/certs/cert.pem"
key_file: "/etc/webhook/certs/key.pem"
http_listen: ":8080"
https_listen: ":8443"
ingress_publish_service: ""
enable_profiling: true
apisix-resource-sync-interval: 1h
kubernetes:
kubeconfig: ""
resync_interval: "6h"
namespace_selector:
- ""
election_id: "ingress-apisix-leader"
ingress_class: "apisix"
ingress_version: "networking/v1"
watch_endpointslices: false
apisix_route_version: "apisix.apache.org/v2"
enable_gateway_api: false
apisix_version: "apisix.apache.org/v2"
plugin_metadata_cm: ""
apisix:
admin_api_version: "v3"
default_cluster_base_url: http://apisix-admin.apisix.svc.cluster.local:9180/apisix/admin
default_cluster_admin_key: "edd1c9f034335f136f87ad84b625c8f1"
default_cluster_name: "default"代码语言：javascript复制vim dp.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: ingress-controller
app.kubernetes.io/version: 1.6.0
name: apisix-ingress-controller
namespace: apisix
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: ingress-controller
template:
metadata:
creationTimestamp: null
labels:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: ingress-controller
spec:
containers:
- command:
- /ingress-apisix/apisix-ingress-controller
- ingress
- --config-path
- /ingress-apisix/conf/config.yaml
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
image: apache/apisix-ingress-controller:1.6.0
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 8080
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: ingress-controller
ports:
- containerPort: 8080
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 8080
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /ingress-apisix/conf
name: configuration
dnsPolicy: ClusterFirst
initContainers:
- command:
- sh
- -c
- until nc -z apisix-admin.apisix.svc.cluster.local 9180 ; do echo
waiting for apisix; sleep 2; done;
image: busybox:1.28
imagePullPolicy: IfNotPresent
name: wait-apisix-admin
resources: {}
securityContext: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: apisix-ingress-controller
serviceAccountName: apisix-ingress-controller
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
items:
- key: config.yaml
path: config.yaml
name: apisix-configmap
name: configuration代码语言：javascript复制vim rbac.yml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: apisix-view-clusterrole
rules:
- apiGroups:
- ""
resources:
- events
verbs:
- "*"
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- persistentvolumeclaims
- pods
- replicationcontrollers
- replicationcontrollers/scale
- serviceaccounts
- services
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- bindings
- limitranges
- namespaces/status
- pods/log
- pods/status
- replicationcontrollers/status
- resourcequotas
- resourcequotas/status
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- list
- watch
- apiGroups:
- apps
resources:
- controllerrevisions
- daemonsets
- deployments
- deployments/scale
- replicasets
- replicasets/scale
- statefulsets
- statefulsets/scale
verbs:
- get
- list
- watch
- apiGroups:
- autoscaling
resources:
- horizontalpodautoscalers
verbs:
- get
- list
- watch
- apiGroups:
- batch
resources:
- cronjobs
- jobs
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- daemonsets
- deployments
- deployments/scale
- ingresses
- networkpolicies
- replicasets
- replicasets/scale
- replicationcontrollers/scale
verbs:
- get
- list
- watch
- apiGroups:
- policy
resources:
- poddisruptionbudgets
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
- ingresses/status
- networkpolicies
verbs:
- '*'
- apiGroups:
- metrics.k8s.io
resources:
- pods
verbs:
- get
- list
- watch
- apiGroups:
- apisix.apache.org
resources:
- apisixroutes
- apisixroutes/status
- apisixupstreams
- apisixupstreams/status
- apisixtlses
- apisixtlses/status
- apisixclusterconfigs
- apisixclusterconfigs/status
- apisixconsumers
- apisixconsumers/status
- apisixpluginconfigs
- apisixpluginconfigs/status
- apisixglobalrules
- apisixglobalrules/status
verbs:
- '*'
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- '*'
- apiGroups:
- discovery.k8s.io
resources:
- endpointslices
verbs:
- get
- list
- watch
- apiGroups:
- gateway.networking.k8s.io
resources:
- httproutes
- tlsroutes
- tcproutes
- gateways
- gatewayclasses
- udproutes
verbs:
- get
- list
- watch
- apiGroups:
- gateway.networking.k8s.io
resources:
- gateways/status
- gatewayclasses/status
verbs:
- get
- update
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: apisix-view-clusterrolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: apisix-view-clusterrole
subjects:
- kind: ServiceAccount
name: apisix-ingress-controller
namespace: apisix
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
name: apisix-ingress-controller
namespace: apisix# 应用下面所有资源配置清单
https://github.com/apache/apisix-ingress-controller/tree/master/samples/deploy/crd/v1代码语言：javascript复制vim svc.yml
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: ingress-controller
app.kubernetes.io/version: 1.6.0
name: apisix-ingress-controller
namespace: apisix
spec:
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- name: http
port: 80
protocol: TCP
targetPort: 8080
selector:
app.kubernetes.io/instance: apisix
app.kubernetes.io/name: ingress-controller
sessionAffinity: None
type: ClusterIP应用资源配置清单
代码语言：javascript复制kubectl apply -f rbac.yml
kubectl apply -f cm.yml
kubectl apply -f dp.yml
kubectl apply -f svc.yml
kubectl apply -f https://raw.githubusercontent.com/apache/apisix-ingress-controller/master/samples/deploy/crd/v1/ApisixConsumer.yaml
kubectl apply -f https://raw.githubusercontent.com/apache/apisix-ingress-controller/master/samples/deploy/crd/v1/ApisixClusterConfig.yaml
kubectl apply -f https://raw.githubusercontent.com/apache/apisix-ingress-controller/master/samples/deploy/crd/v1/ApisixGlobalRule.yaml
kubectl apply -f https://raw.githubusercontent.com/apache/apisix-ingress-controller/master/samples/deploy/crd/v1/ApisixPluginConfig.yaml
kubectl apply -f https://raw.githubusercontent.com/apache/apisix-ingress-controller/master/samples/deploy/crd/v1/ApisixRoute.yaml
kubectl apply -f https://raw.githubusercontent.com/apache/apisix-ingress-controller/master/samples/deploy/crd/v1/ApisixTls.yaml
kubectl apply -f https://raw.githubusercontent.com/apache/apisix-ingress-controller/master/samples/deploy/crd/v1/ApisixUpstream.yamlAPISIX-Dashboard
准备资源配置清单
ConfigMapDeploymentService
代码语言：javascript复制vim cm.yml
apiVersion: v1
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/instance: apisix-dashboard
app.kubernetes.io/name: apisix-dashboard
app.kubernetes.io/version: 3.0.0
name: apisix-dashboard
namespace: apisix
data:
conf.yaml: |-
conf:
listen:
host: 0.0.0.0
port: 9000
etcd:
prefix: "/apisix"
endpoints:
- https://10.1.1.100:2379
- https://10.1.1.120:2379
- https://10.1.1.130:2379
mtls:
key_file: /etcd-ssl/tls.key
cert_file: /etcd-ssl/tls.crt
ca_file: /usr/local/apisix-dashboard/ssl/ca.pem
log:
error_log:
level: warn
file_path: /dev/stderr
access_log:
file_path: /dev/stdout
authentication:
secret: secret
expire_time: 3600
users:
- username: admin # 配置登陆用户名
password: admin # 登陆密码代码语言：javascript复制vim dp.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/instance: apisix-dashboard
app.kubernetes.io/name: apisix-dashboard
app.kubernetes.io/version: 3.0.0
name: apisix-dashboard
namespace: apisix
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/instance: apisix-dashboard
app.kubernetes.io/name: apisix-dashboard
template:
metadata:
labels:
app.kubernetes.io/instance: apisix-dashboard
app.kubernetes.io/name: apisix-dashboard
spec:
containers:
- image: apache/apisix-dashboard:3.0.0-alpine
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /ping
port: http
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: apisix-dashboard
ports:
- containerPort: 9000
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /ping
port: http
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
securityContext: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /usr/local/apisix-dashboard/conf/conf.yaml
name: apisix-dashboard-config
subPath: conf.yaml
- mountPath: /etcd-ssl/
name: apisix-dashboard-etcd
- mountPath: /usr/local/apisix-dashboard/ssl/ca.pem
name: apisix-dashboard-etcd-ca
subPath: ca.pem
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
name: apisix-dashboard
name: apisix-dashboard-config
- secret:
secretName: etcd-tls
name: apisix-dashboard-etcd
- secret:
secretName: apisix-ca
name: apisix-dashboard-etcd-ca代码语言：javascript复制vim svc.yml
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/instance: apisix-dashboard
app.kubernetes.io/name: apisix-dashboard
app.kubernetes.io/version: 3.0.0
name: apisix-dashboard
namespace: apisix
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
nodePort: 31000
selector:
app.kubernetes.io/instance: apisix-dashboard
app.kubernetes.io/name: apisix-dashboard
type: NodePort应用资源配置清单
代码语言：javascript复制kubectl apply -f cm.yml
kubectl apply -f dp.yml
kubectl apply -f svc.ymlApisixRoute
现在我们可以为 Dashboard 创建一个路由规则，新建一个如下所示的 ApisixRoute 资源对象即可：
cat > dashboard-ing.yml <<EOF
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
name: dashboard
namespace: apisix
spec:
http:
- name: root
match:
hosts:
- apisix.od.com
paths:
- '/*'
backends:
- serviceName: apisix-dashboard
servicePort: 80
EOF创建后 apisix-ingress-controller 会将上面的资源对象通过 admin api 映射成 APISIX 中的配置：
kubectl apply -f dashboard-ing.yml
kubectl get apisixroutes.apisix.apache.org -A
NAMESPACE NAME HOSTS URIS AGE
apisix dashboard ["apisix.od.com"] ["/*"] 3m30s可以在本地测试的时候可以使用 kubectl port-forward 将服务暴露在节点的 80 端口上：
# node2 节点暴露 apisix-gateway 服务
kubectl port-forward --address 0.0.0.0 svc/apisix-gateway 80:80 443:443 -n apisix本次我使用nginx做的转发。
打开http://apisix.od.com
默认登录用户名和密码都是 admin，登录后在路由菜单下正常可以看到上面我们创建的这个 dashboard 的路由信息：
使用 APISIX，也一定要理解其中的路由 Route 这个概念，路由（Route）是请求的入口点，它定义了客户端请求与服务之间的匹配规则，路由可以与服务（Service）、上游（Upstream）关联，一个服务可对应一组路由，一个路由可以对应一个上游对象（一组后端服务节点），因此，每个匹配到路由的请求将被网关代理到路由绑定的上游服务中。
