Istio Traffic Management

理解Istio流量管理

参考官网：https://istio.io/zh/docs/tasks/traffic-management/

Istio核心概念：

  • Gateway定义服务出入口
  • Virtual Service定义服务路由策略
  • Destination Rule定义服务子集、负载均衡、连接池等信息

Envoy核心概念：

  • Listener，监听器，命名网地址可以被下游客户端连接，由LDS提供动态配置接口
  • Router，路由，一组将虚拟主机(virtual hosts)与群集(cluster)匹配的规则(rule)，允许创建流量转移规则，由RDS提供动态配置接口
  • Cluster，集群，指Envoy连接到的逻辑上相同的一组上游主机，由CDS提供动态配置接口
  • Endpoint，端点，群集(Cluster)中可用的IP和端口，由EDS提供动态配置接口

原理：Istio配置下发后，转换为Envoy配置后在proxy上生效，实现Istio策略下发，Envoy流量控制，完成整体服务网格流量管理

网关

设置服务集、路由

$ kubectl apply -f samples/bookinfo/networking/destination-rule-all.yaml -n book
destinationrule.networking.istio.io/productpage created
destinationrule.networking.istio.io/reviews created
destinationrule.networking.istio.io/ratings created
destinationrule.networking.istio.io/details created
$ kubectl apply -f samples/bookinfo/networking/virtual-service-all-v1.yaml -n book
virtualservice.networking.istio.io/productpage created
virtualservice.networking.istio.io/reviews created
virtualservice.networking.istio.io/ratings created
virtualservice.networking.istio.io/details created
  • destination-rule-all.yaml，根据pod的标签version值划分服务子集
  • virtual-service-all-v1.yaml，定义路径策略，将各服务请求默认路由至v1版本

Virtual Service如果关联了Gateway则同时在具体ingressgateway的proxy上生效，用于服务入口控制，例如productpage

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: productpage
  ...
spec:
  gateways:
  - bookinfo-gateway
  - mesh
  hosts:
  - productpage
  http:
  - route:
    - destination:
        host: productpage
        subset: v1

$ istioctl proxy-config route reviews-v1-64bc5454b9-6d96p -n book -o json --name 9080
......
                        "route": {
                            "cluster": "outbound|9080|v1|details.book.svc.cluster.local",
......
                        "route": {
                            "cluster": "outbound|9080|v1|productpage.book.svc.cluster.local",
......
                        "route": {
                            "cluster": "outbound|9080|v1|ratings.book.svc.cluster.local",
......
                        "route": {
                            "cluster": "outbound|9080|v1|reviews.book.svc.cluster.local",
......

在所有proxy节点上都能看到将所有服务9080端口的访问均路由至v1版本的cluster中

Destination Rule配置服务子集、负载均衡、连接池等信息，

$ istioctl proxy-config cluster reviews-v1-64bc5454b9-6d96p -n book --fqdn book.svc.cluster.local
SERVICE FQDN                           PORT     SUBSET          DIRECTION     TYPE
details.book.svc.cluster.local         9080     -               outbound      EDS
details.book.svc.cluster.local         9080     v1              outbound      EDS
details.book.svc.cluster.local         9080     v2              outbound      EDS
productpage.book.svc.cluster.local     9080     -               outbound      EDS
productpage.book.svc.cluster.local     9080     v1              outbound      EDS
ratings.book.svc.cluster.local         9080     -               outbound      EDS
ratings.book.svc.cluster.local         9080     v1              outbound      EDS
ratings.book.svc.cluster.local         9080     v2              outbound      EDS
reviews.book.svc.cluster.local         9080     -               outbound      EDS
reviews.book.svc.cluster.local         9080     http            inbound       STATIC
reviews.book.svc.cluster.local         9080     v1              outbound      EDS
reviews.book.svc.cluster.local         9080     v2              outbound      EDS
reviews.book.svc.cluster.local         9080     v3              outbound      EDS

可以看到Envoy为每一个服务都根据version划分了子集

$ istioctl proxy-config cluster reviews-v1-64bc5454b9-6d96p -n book --fqdn reviews.book.svc.cluster.local -o json
......
        "name": "outbound|9080|v1|reviews.book.svc.cluster.local",
        "type": "EDS",
        "edsClusterConfig": {
            "edsConfig": {
                "ads": {}
            },
            "serviceName": "outbound|9080|v1|reviews.book.svc.cluster.local"
        },
......

指定v1版本的路由后端为outbound|9080|v1|reviews.book.svc.cluster.local

$ istioctl proxy-config endpoint reviews-v1-64bc5454b9-6d96p -n book -o json --cluster "outbound|9080|v1|reviews.book.svc.cluster.local"
[
    {
        "name": "outbound|9080|v1|reviews.book.svc.cluster.local",
        "addedViaApi": true,
        "hostStatuses": [
            {
                "address": {
                    "socketAddress": {
                        "address": "10.1.0.149",
                        "portValue": 9080
                    }
                },

定义v1版本的后端节点信息列表

至此，完成服务从访问到具体pod节点的路由分发流量控制

http://localhost:5678/productpage，再次访问样例，将只会看到所有服务v1版本的内容。

alt

入口流量

所有入口流量都转发到Envoy的15006端口

Listener:

$ istioctl pc listener productpage-v1-76589d9fdc-4krcv -n book -o json --port 15006
[
    {
        "name": "virtualInbound",
        "address": {
            "socketAddress": {
                "address": "0.0.0.0",
                "portValue": 15006
            }
        },
......
                "filterChainMatch": {
                    "destinationPort": 9080,
......
                "filters": [
                    {
                        "name": "envoy.filters.network.metadata_exchange",
                        "config": {
                            "protocol": "istio-peer-exchange"
                        }
                    },
                    {
                        "name": "envoy.http_connection_manager",
                        "typedConfig": {
                            "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager",
                            "statPrefix": "inbound_10.1.0.199_9080",
                            "routeConfig": {
                                "name": "inbound|9080|http|productpage.book.svc.cluster.local",
                                "virtualHosts": [
                                    {
                                        "name": "inbound|http|9080",
                                        "domains": [
                                            "*"
                                        ],
                                        "routes": [
                                            {
                                                "name": "default",
                                                "match": {
                                                    "prefix": "/"
                                                },
                                                "route": {
                                                    "cluster": "inbound|9080|http|productpage.book.svc.cluster.local",
                                                    "timeout": "0s",
                                                    "maxGrpcTimeout": "0s"
                                                },
                                                "decorator": {
                                                    "operation": "productpage.book.svc.cluster.local:9080/*"
                                                }
                                            }
                                        ]
                                    }
                                ],
                                "validateClusters": false
                            },
......

到9080端口（业务端口），采用路由配置"inbound|9080|http|productpage.book.svc.cluster.local”，路由到集群"inbound|9080|http|productpage.book.svc.cluster.local”

Cluster：

$ istioctl pc cluster productpage-v1-76589d9fdc-4krcv -n book -o json --fqdn productpage.book.svc.cluster.local --port 9080 --direction inbound
[
    {
        "name": "inbound|9080|http|productpage.book.svc.cluster.local",
        "type": "STATIC",
        "connectTimeout": "1s",
        "loadAssignment": {
            "clusterName": "inbound|9080|http|productpage.book.svc.cluster.local",
            "endpoints": [
                {
                    "lbEndpoints": [
                        {
                            "endpoint": {
                                "address": {
                                    "socketAddress": {
                                        "address": "127.0.0.1",
                                        "portValue": 9080
                                    }
                                }
                            }
                        }
                    ]
                }
            ]
        },
......
]

后端服务节点地址为127.0.0.1，即Productpage业务地址

流程总结：

  • 入口流量被iptables规则拦截至15006的Envoy代理监听端口
  • Envoy监听15006端口入口流量，关联路由配置"inbound|9080|http|productpage.book.svc.cluster.local”
  • 路由配置定义服务集群"inbound|9080|http|productpage.book.svc.cluster.local”
  • 服务集群提供后端节点"127.0.0.1”
  • Pod内部通过本机地址和服务端口和业务容器进行交互

出口流量

所有出口流量都转发到Envoy的15001端口

Productpage需要访问Reviews子服务内容

Listener:

$ istioctl pc listener productpage-v1-76589d9fdc-4krcv -n book -o json --port 15001
[
    {
        "name": "virtualOutbound",
        "address": {
            "socketAddress": {
                "address": "0.0.0.0",
                "portValue": 15001
            }
        },
......
        "useOriginalDst": true,
        "trafficDirection": "OUTBOUND"
    }
]

进入15001的流量，Envoy没有做过多处理，“useOriginalDst”: true表示转发给其他Listener处理，当前review服务端口为9080

$ istioctl pc listener productpage-v1-76589d9fdc-4krcv -n book -o json --port 9080
......
    {
        "name": "0.0.0.0_9080",
        "address": {
            "socketAddress": {
                "address": "0.0.0.0",
                "portValue": 9080
            }
        },
......
                "filters": [
                    {
                        "name": "envoy.http_connection_manager",
                        "typedConfig": {
                            "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager",
                            "statPrefix": "outbound_0.0.0.0_9080",
                            "rds": {
                                "configSource": {
                                    "ads": {}
                                },
                                "routeConfigName": "9080"
                            },
......
        "trafficDirection": "OUTBOUND"
    }
]

转发至9080的流量，关联至路由配置"9080”，rds为动态路由配置，即通过Istio获取的配置

Route：

$ istioctl pc route productpage-v1-76589d9fdc-4krcv -n book -o json --name 9080
[
    {
        "name": "9080",
......
            {
                "name": "reviews.book.svc.cluster.local:9080",
                "domains": [
                    "reviews.book.svc.cluster.local",
                    "reviews.book.svc.cluster.local:9080",
                    "reviews",
                    "reviews:9080",
                    "reviews.book.svc.cluster",
                    "reviews.book.svc.cluster:9080",
                    "reviews.book.svc",
                    "reviews.book.svc:9080",
                    "reviews.book",
                    "reviews.book:9080",
                    "10.101.136.221",
                    "10.101.136.221:9080"
                ],
                "routes": [
                    {
                        "match": {
                            "prefix": "/"
                        },
                        "route": {
                            "cluster": "outbound|9080|v1|reviews.book.svc.cluster.local",
                            "timeout": "0s",
                            "retryPolicy": {
                                "retryOn": "connect-failure,refused-stream,unavailable,cancelled,resource-exhausted,retriable-status-codes",
                                "numRetries": 2,
                                "retryHostPredicate": [
                                    {
                                        "name": "envoy.retry_host_predicates.previous_hosts"
                                    }
                                ],
                                "hostSelectionRetryMaxAttempts": "5",
                                "retriableStatusCodes": [
                                    503
                                ]
                            },
                            "maxGrpcTimeout": "0s"
                        },
......

访问reviews的请求，被路由至集群"outbound|9080|v1|reviews.book.svc.cluster.local"中

Cluster：

$ istioctl pc cluster productpage-v1-76589d9fdc-4krcv -n book -o json --fqdn reviews.book.svc.cluster.local --port 9080 --direction outbound
......
        "name": "outbound|9080|v1|reviews.book.svc.cluster.local",
        "type": "EDS",
        "edsClusterConfig": {
            "edsConfig": {
                "ads": {}
            },
            "serviceName": "outbound|9080|v1|reviews.book.svc.cluster.local"
        },
        "connectTimeout": "1s",
        "circuitBreakers": {
            "thresholds": [
                {
                    "maxConnections": 4294967295,
                    "maxPendingRequests": 4294967295,
                    "maxRequests": 4294967295,
                    "maxRetries": 4294967295
                }
            ]
        },
......

集群定义后端节点列表"outbound|9080|v1|reviews.book.svc.cluster.local”，eds为动态节点配置，即通过Istio获取的配置

Endpoint：

$ istioctl pc endpoint productpage-v1-76589d9fdc-4krcv -n book -o json --cluster "outbound|9080|v1|reviews.book.svc.cluster.local"
[
    {
        "name": "outbound|9080|v1|reviews.book.svc.cluster.local",
        "addedViaApi": true,
        "hostStatuses": [
            {
                "address": {
                    "socketAddress": {
                        "address": "10.1.0.196",
                        "portValue": 9080
                    }
                },
                "stats": [
                    {
                        "name": "cx_connect_fail"
                    },
                    {
                        "name": "cx_total"
                    },
                    {
                        "name": "rq_error"
                    },
                    {
                        "name": "rq_success"
                    },
                    {
                        "name": "rq_timeout"
                    },
                    {
                        "name": "rq_total"
                    },
                    {
                        "type": "GAUGE",
                        "name": "cx_active"
                    },
                    {
                        "type": "GAUGE",
                        "name": "rq_active"
                    }
                ],
                "healthStatus": {
                    "edsHealthStatus": "HEALTHY"
                },
                "weight": 1
            }
        ]
    }
]

获取后端节点列表信息，即reviews服务v1版本的实例列表信息，完成出口流量的转发

$ kubectl get pod -n book -o wide
NAME                              READY   STATUS    RESTARTS   AGE   IP           NODE             NOMINATED NODE   READINESS GATES
details-v1-74f858558f-fh72x       2/2     Running   0          29h   10.1.0.194   docker-desktop   <none>           <none>
productpage-v1-76589d9fdc-4krcv   2/2     Running   0          29h   10.1.0.199   docker-desktop   <none>           <none>
ratings-v1-7855f5bcb9-kc6zg       2/2     Running   0          29h   10.1.0.195   docker-desktop   <none>           <none>
reviews-v1-64bc5454b9-wk9vg       2/2     Running   0          29h   10.1.0.196   docker-desktop   <none>           <none>
reviews-v2-76c64d4bdf-q4js5       2/2     Running   0          29h   10.1.0.197   docker-desktop   <none>           <none>
reviews-v3-5545c7c78f-9r9lc       2/2     Running   0          29h   10.1.0.198   docker-desktop   <none>           <none>

流程总结：

  • 出口流量被iptables规则拦截至15001的Envoy代理监听端口
  • Envoy将15001监听转发给目标监听器，即9080
  • Envoy监听9080端口出口流量，关联路由配置"outbound|9080|v1|reviews.book.svc.cluster.local”
  • 路由配置定义服务集群"outbound|9080|v1|reviews.book.svc.cluster.local”
  • 服务集群提供后端节点"10.1.0.196”，即远端reviews服务节点

通用规则

规则 Istio Envoy
请求路由 VirtualService route
基于用户身份的路由 VirtualService -> match route -> match
故障注入，延迟 VirtualService -> fault -> delay route -> typedPerFilterConfig -> envoy.fault -> delay
故障注入, 终止 VirtualService -> fault -> abort route -> typedPerFilterConfig -> envoy.fault -> abort
流量转移，基于权重 VirtualService -> route -> weight route -> weightedClusters
超时 VirtualService -> route -> timeout route -> timeout
熔断 DestinationRule -> trafficPolicy cluster -> circuitBreakers

Istio和Envoy配置对应关系：

  • Gateway <-> Listener
  • VirtualService <-> Route
  • DestinationRule <-> Cluster

Istio流量管理配置参考：https://istio.io/zh/docs/reference/config/networking/

Envoy配置参考：https://www.envoyproxy.io/docs/envoy/latest/configuration/configuration