理解Envoy配置文件
Istio核心功能都是通过渲染Envoy配置文件实现的，所以很有必要看下Envoy配置文件具体含义
Envoy数据流向：Downstream -> Listener -> Route -> Cluster -> Endpoint -> Upstream
Envoy配置文件主要包括以下几部分：
- bootstrap，启动时配置
- listener，监听器配置
- route，路由规则配置
- cluster，服务集群配置
- endpoint，后端节点配置
所有配置都包括静态配置和动态配置2大部分
除boostrap外，其他配置对应Envoy数据流的每个阶段，前一阶段关联至后一阶段，完成整个数据流配置
bootstrap
Envoy启动时加载的配置的信息，通过 sidecar 启动参数注入的
$ istioctl pc bootstrap productpage-v1-76589d9fdc-4krcv -n book -o json
{
"bootstrap": {
"node": {
"id": "sidecar~10.1.0.199~productpage-v1-76589d9fdc-4krcv.book~book.svc.cluster.local",
"cluster": "productpage.book",
......运行节点信息
"staticResources": {
"listeners": [
{
"address": {
"socketAddress": {
"address": "0.0.0.0",
"portValue": 15090
}
},
"filterChains": [
{
"filters": [
{
"name": "envoy.http_connection_manager",
......prometheus监控接口信息
"clusters": [
{
"name": "prometheus_stats",
......对接prometheus监控系统信息
{
"name": "sds-grpc",
"type": "STATIC",
"connectTimeout": "10s",
......对接SDS密钥发现服务信息
{
"name": "xds-grpc",
"type": "STRICT_DNS",
"connectTimeout": "10s",
......对接XDS配置发现服务信息
{
"name": "zipkin",
"type": "STRICT_DNS",
"connectTimeout": "1s",
......对接zipkin调用跟踪系统信息
"dynamicResources": {
"ldsConfig": {
"ads": {
}
},
"cdsConfig": {
"ads": {
}
},
"adsConfig": {
"apiType": "GRPC",
"grpcServices": [
{
"envoyGrpc": {
"clusterName": "xds-grpc"
}
}
]
}
},
......对接动态配置发现服务信息
"statsConfig": {
......监控指标配置
"tracing": {
"http": {
"name": "envoy.zipkin",
......zipkin调用跟踪信息
"admin": {
"accessLogPath": "/dev/null",
"address": {
"socketAddress": {
"address": "127.0.0.1",
"portValue": 15000
}
}
}
......管理入口配置
Listener
监听器配置，可以有多个监控IP和端口，根据关联的Route策略进行请求转发
$ kubectl get svc -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
book-service LoadBalancer 10.105.160.227 localhost 5678:32762/TCP
grafana ClusterIP 10.101.202.54 <none> 3000/TCP
istio-egressgateway ClusterIP 10.107.80.98 <none> 80/TCP,443/TCP,15443/TCP
istio-ingressgateway LoadBalancer 10.102.217.109 localhost 15020:30874/TCP,80:32114/TCP,4
istio-pilot ClusterIP 10.97.130.47 <none> 15010/TCP,15011/TCP,15012/TCP,
istiod ClusterIP 10.109.209.152 <none> 15012/TCP,443/TCP
jaeger-agent ClusterIP None <none> 5775/UDP,6831/UDP,6832/UDP
jaeger-collector ClusterIP 10.108.46.238 <none> 14267/TCP,14268/TCP,14250/TCP
jaeger-collector-headless ClusterIP None <none> 14250/TCP
jaeger-query ClusterIP 10.106.235.0 <none> 16686/TCP
kiali ClusterIP 10.108.137.51 <none> 20001/TCP
prometheus ClusterIP 10.100.161.1 <none> 9090/TCP
tracing ClusterIP 10.105.1.246 <none> 80/TCP
zipkin ClusterIP 10.109.100.21 <none> 9411/TCP
$ kubectl get pod -o wide -n book
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
details-v1-74f858558f-fh72x 2/2 Running 0 46h 10.1.0.194 docker-desktop <none> <none>
productpage-v1-76589d9fdc-4krcv 2/2 Running 0 46h 10.1.0.199 docker-desktop <none> <none>
ratings-v1-7855f5bcb9-kc6zg 2/2 Running 0 46h 10.1.0.195 docker-desktop <none> <none>
reviews-v1-64bc5454b9-wk9vg 2/2 Running 0 46h 10.1.0.196 docker-desktop <none> <none>
reviews-v2-76c64d4bdf-q4js5 2/2 Running 0 46h 10.1.0.197 docker-desktop <none> <none>
reviews-v3-5545c7c78f-9r9lc 2/2 Running 0 46h 10.1.0.198 docker-desktop <none> <none>
$ istioctl pc listener productpage-v1-76589d9fdc-4krcv -n book
ADDRESS PORT TYPE
10.1.0.199 9080 HTTP //Receives all inbound traffic on 9080 from listener 0.0.0.0_15006
10.1.0.199 15020 TCP //mgmtCluster
10.97.130.47 15011 TCP //istio-pilot
10.97.130.47 443 TCP //istio-pilot
10.102.217.109 443 TCP //istio-ingressgateway
10.102.217.109 15443 TCP //istio-ingressgateway
10.107.80.98 15443 TCP //istio-egressgateway
10.97.130.47 15012 TCP //istio-pilot
10.96.0.10 53 TCP //kube-dns
10.109.209.152 15012 TCP //istiod
10.109.209.152 443 TCP //istiod
10.102.217.109 31400 TCP //istio-ingressgateway
10.96.0.1 443 TCP //kubernetes
10.107.80.98 443 TCP //istio-egressgateway
10.102.217.109 15031 TCP //istio-ingressgateway
0.0.0.0 3000 TCP
10.96.0.10 9153 TCP //kube-dns
0.0.0.0 8080 TCP
0.0.0.0 9090 TCP
0.0.0.0 9411 TCP
0.0.0.0 80 TCP
0.0.0.0 20001 TCP
10.102.217.109 15029 TCP //istio-ingressgateway
10.107.78.115 443 TCP
10.102.217.109 15032 TCP //istio-ingressgateway
10.102.217.109 15020 TCP //istio-ingressgateway
0.0.0.0 9080 TCP
0.0.0.0 15010 TCP
0.0.0.0 15014 TCP
10.102.217.109 15030 TCP //istio-ingressgateway
0.0.0.0 15001 TCP //Receives all outbound traffic to the pod from IP tables and hands over to virtual listener
0.0.0.0 15006 TCP //Receives all inbound traffic to the pod from IP tables and hands over to virtual listener
0.0.0.0 5678 TCP
10.108.46.238 14268 TCP //jaeger-collector
10.108.46.238 14267 TCP //jaeger-collector
0.0.0.0 14250 TCP
10.106.235.0 16686 TCP //jaeger-query
10.108.46.238 14250 TCP //jaeger-collector
0.0.0.0 15090 HTTP
Listener分几大类：
- 控制面服务Istio
- 总入口15006
- 总出口15001
- 自身服务入口，9080/HTTP
- 不带IP的出口
以带IP的出口0.0.0.0/9080/TCP为例
$ istioctl pc listener productpage-v1-76589d9fdc-4krcv -n book -o json
......
{
"name": "0.0.0.0_9080", //名称，唯一限定名
"address": { //具体监听IP和端口
"socketAddress": {
"address": "0.0.0.0",
"portValue": 9080
}
},
"filterChains": [ //顺序执行的过滤器，Envoy内置了一些通用的filter
{
"filterChainMatch": { //匹配条件，匹配成功的请求，走下面filters，默认丢弃本机IP请求，防止请求死循环
......
},
"filters": [ //filter列表
......
{
"filters": [
{
"name": "envoy.http_connection_manager", //关键filter，http连接管理
"typedConfig": {
"@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager",
"statPrefix": "outbound_0.0.0.0_9080",
"rds": { //RDS动态路由信息配置
"configSource": {
"ads": {}
},
"routeConfigName": "9080" //关联的具体路由策略
},
"httpFilters": [ //http filter列表，主要扩展点，采集请求属性信息
......
],
"tracing": { //请求跟踪采样率配置
"clientSampling": {
"value": 100
},
"randomSampling": {
"value": 100
},
"overallSampling": {
"value": 100
}
},
"streamIdleTimeout": "0s",
"accessLog": [ //请求访问日志配置
{
"name": "envoy.file_access_log",
"typedConfig": {
"@type": "type.googleapis.com/envoy.config.accesslog.v2.FileAccessLog",
"path": "/dev/stdout",
"format": "[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% \"%DYNAMIC_METADATA(istio.mixer:status)%\" \"%UPSTREAM_TRANSPORT_FAILURE_REASON%\" %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\" %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %REQUESTED_SERVER_NAME% %ROUTE_NAME%\n"
}
}
],
"useRemoteAddress": false,
"generateRequestId": true,
"upgradeConfigs": [
{
"upgradeType": "websocket"
}
],
"normalizePath": true
}
}
]
},
......
"trafficDirection": "OUTBOUND" //请求的流向
},
Listen根据RDS的routeConfigName，“9080"关联到具体的路由策略
Route
路由策略配置信息
$ istioctl pc route productpage-v1-76589d9fdc-4krcv -n book -o json
......
{
"name": "9080", //名称
"virtualHosts": [
{
"name": "allow_any",
"domains": [
"*"
],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "PassthroughCluster", //定义默认策略，透传到具体服务集群
"timeout": "0s"
}
}
]
},
......
{
"name": "details.book.svc.cluster.local:9080", //具体服务集群路由配置，默认Envoy在当前节点上包括所有服务集群的配置信息
"domains": [ //具体域名匹配规则
"details.book.svc.cluster.local",
"details.book.svc.cluster.local:9080",
"details",
"details:9080",
"details.book.svc.cluster",
"details.book.svc.cluster:9080",
"details.book.svc",
"details.book.svc:9080",
"details.book",
"details.book:9080",
"10.107.66.150",
"10.107.66.150:9080"
],
"routes": [ //具体路由策略
{
"match": { //url路径匹配
"prefix": "/"
},
"route": { //路由配置
"cluster": "outbound|9080|v1|details.book.svc.cluster.local", //地址
"timeout": "0s", //超时
"retryPolicy": { //重试
"retryOn": "connect-failure,refused-stream,unavailable,cancelled,resource-exhausted,retriable-status-codes",
"numRetries": 2,
"retryHostPredicate": [
{
"name": "envoy.retry_host_predicates.previous_hosts"
}
],
"hostSelectionRetryMaxAttempts": "5",
"retriableStatusCodes": [
503
]
},
"maxGrpcTimeout": "0s"
},
"metadata": {
"filterMetadata": {
"istio": {
"config": "/apis/networking.istio.io/v1alpha3/namespaces/book/virtual-service/details"
}
}
},
"decorator": {
"operation": "details.book.svc.cluster.local:9080/*"
}
}
]
},
......
Route根据cluster指定服务集群，outbound|9080|v1|details.book.svc.cluster.local
Cluster
具体服务集群配置信息
$ istioctl pc cluster productpage-v1-76589d9fdc-4krcv -n book -o json
......
{
"transportSocketMatches": [
{
"name": "tlsMode-istio", //TLS配置
"match": {
"tlsMode": "istio"
},
"transportSocket": {
"name": "envoy.transport_sockets.tls",
"typedConfig": {
"@type": "type.googleapis.com/envoy.api.v2.auth.UpstreamTlsContext",
"commonTlsContext": {
"tlsCertificateSdsSecretConfigs": [ //从SDS获取证书密钥等信息
{
"name": "default",
"sdsConfig": {
"apiConfigSource": {
"apiType": "GRPC",
"grpcServices": [
{
"envoyGrpc": {
"clusterName": "sds-grpc"
}
}
]
}
}
}
],
"combinedValidationContext": {
"defaultValidationContext": {
"verifySubjectAltName": [
"spiffe://cluster.local/ns/book/sa/bookinfo-details" //通过SPIFFE证书校验信息
]
},
"validationContextSdsSecretConfig": {
"name": "ROOTCA",
"sdsConfig": {
"apiConfigSource": {
"apiType": "GRPC",
"grpcServices": [
{
"envoyGrpc": {
"clusterName": "sds-grpc"
}
}
]
}
}
}
},
"alpnProtocols": [
"istio-peer-exchange",
"istio"
]
},
"sni": "outbound_.9080_.v1_.details.book.svc.cluster.local"
}
}
},
{
"name": "tlsMode-disabled",
"match": {},
"transportSocket": {
"name": "envoy.transport_sockets.raw_buffer"
}
}
],
"name": "outbound|9080|v1|details.book.svc.cluster.local", //名称
"type": "EDS", //类型，EDS为后端节点动态配置
"edsClusterConfig": {
"edsConfig": {
"ads": {}
},
"serviceName": "outbound|9080|v1|details.book.svc.cluster.local" //关联至具体后端集群
},
"connectTimeout": "1s",
"circuitBreakers": { //熔断配置
"thresholds": [
{
"maxConnections": 4294967295,
"maxPendingRequests": 4294967295,
"maxRequests": 4294967295,
"maxRetries": 4294967295
}
]
},
"metadata": {
"filterMetadata": {
"istio": {
"config": "/apis/networking.istio.io/v1alpha3/namespaces/book/destination-rule/details",
"subset": "v1"
}
}
},
"filters": [
{
"name": "envoy.filters.network.upstream.metadata_exchange",
"typedConfig": {
"@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
"typeUrl": "type.googleapis.com/envoy.tcp.metadataexchange.config.MetadataExchange",
"value": {
"protocol": "istio-peer-exchange"
}
}
}
]
},
Cluster根据serviceName获取后端节点列表信息，outbound|9080|v1|details.book.svc.cluster.local
Endpoint
具体提供服务的后端实例节点列表信息
$ istioctl pc endpoint productpage-v1-76589d9fdc-4krcv -n book -o json
......
{
"name": "outbound|9080|v1|details.book.svc.cluster.local", //名称
"addedViaApi": true,
"hostStatuses": [ //主机列表和状态
{
"address": { //具体IP地址和端口
"socketAddress": {
"address": "10.1.0.194",
"portValue": 9080
}
},
"stats": [ //状态统计信息
{
"name": "cx_connect_fail"
},
{
"name": "cx_total"
},
{
"name": "rq_error"
},
{
"name": "rq_success"
},
{
"name": "rq_timeout"
},
{
"name": "rq_total"
},
{
"type": "GAUGE",
"name": "cx_active"
},
{
"type": "GAUGE",
"name": "rq_active"
}
],
"healthStatus": { //监控状态
"edsHealthStatus": "HEALTHY"
},
"weight": 1 //具体权重
}
]
},
......
具体详细配置项，参考Envoy v2 API reference
参考：