在kratos源码分析系列(1)介绍完基本使用后,我们分目录介绍下它的源码实现
1,api目录
首先看下api目录,它实现了提供服务接口元信息的能力 proto定义位于:kratos/api/metadata/metadata.proto
代码语言:javascript复制// Metadata is api definition metadata service.
service Metadata {
// ListServices list the full name of all services.
rpc ListServices (ListServicesRequest) returns (ListServicesReply) {
option (google.api.http) = {
get: "/services",
};
}
// GetServiceDesc get the full fileDescriptorSet of service.
rpc GetServiceDesc (GetServiceDescRequest) returns (GetServiceDescReply) {
option (google.api.http) = {
get: "/services/{name}",
};
}
}
它本质上是grpc reflection 功能的一个包装,不同的是它同时支持了http服务,生成了对应go、http、grpc代码,server的实现逻辑位于
kratos/api/metadata/server.go
代码语言:javascript复制func (s *Server) ListServices(ctx context.Context, in *ListServicesRequest) (*ListServicesReply, error) {
if err := s.load(); err != nil {
for name := range s.services {
reply.Services = append(reply.Services, name)
}
for name, methods := range s.methods {
for _, method := range methods {
reply.Methods = append(reply.Methods, fmt.Sprintf("/%s/%s", name, method))
}
}
代码语言:javascript复制func (s *Server) load() error {
for name, info := range s.srv.GetServiceInfo() {
获取服务元信息最终调用了google.golang.org/grpc@v1.46.2/server.go,服务的元信息是注册服务的时候写入的
代码语言:javascript复制func (s *Server) GetServiceInfo() map[string]ServiceInfo {
代码语言:javascript复制type Server struct {
services map[string]*serviceInfo
代码语言:javascript复制func (s *Server) RegisterService(sd *ServiceDesc, ss interface{}) {
s.register(sd, ss)
代码语言:javascript复制func (s *Server) register(sd *ServiceDesc, ss interface{}) {
s.services[sd.ServiceName] = info
fd, err := parseMetadata(info.Metadata)
protoSet, err := allDependency(fd)
,获取元信息的时候,就是解析proto文件,返回方法名和对应元信息
代码语言:javascript复制func allDependency(fd *dpb.FileDescriptorProto) ([]*dpb.FileDescriptorProto, error) {
for _, dep := range fd.Dependency {
fdDep, err := fileDescriptorProto(dep)
代码语言:javascript复制func fileDescriptorProto(path string) (*dpb.FileDescriptorProto, error) {
fd, err := protoregistry.GlobalFiles.FindFileByPath(path)
fdpb := protodesc.ToFileDescriptorProto(fd)
最终实现位于:google.golang.org/protobuf@v1.28.0/reflect/protoregistry/registry.go
代码语言:javascript复制func (r *Files) FindFileByPath(path string) (protoreflect.FileDescriptor, error) {
if r == nil {
return nil, NotFound
}
if r == GlobalFiles {
globalMutex.RLock()
defer globalMutex.RUnlock()
}
fds := r.filesByPath[path]
2,cmd目录
cmd目录实现了三个代码生成工具,分别对应krotos命令、生成error的protoc插件、生成http的protoc插件。
kratos命令是一系列子命令的集合cmd/kratos/main.go
代码语言:javascript复制func init() {
rootCmd.AddCommand(project.CmdNew)
rootCmd.AddCommand(proto.CmdProto)
rootCmd.AddCommand(upgrade.CmdUpgrade)
rootCmd.AddCommand(change.CmdChange)
rootCmd.AddCommand(run.CmdRun)
}
代码语言:javascript复制func main() {
if err := rootCmd.Execute()
在base里包装了常用的go命令,比如go install等cmd/kratos/internal/base/install_compatible.go
代码语言:javascript复制func GoInstall(path ...string) error {
for _, p := range path {
fmt.Printf("go get -u %sn", p)
cmd := exec.Command("go", "get", "-u", p)
cmd/kratos/internal/base/install.go 对一些没有指定版本的依赖,默认拉最新的。
代码语言:javascript复制func GoInstall(path ...string) error {
for _, p := range path {
if !strings.Contains(p, "@") {
p = "@latest"
cmd := exec.Command("go", "install", p)
cmd/kratos/internal/base/mod.go获取go mod文件的路径,并解析go mod文件,最终调用了golang扩展包的modfile
代码语言:javascript复制func ModulePath(filename string) (string, error) {
modBytes, err := os.ReadFile(filename)
return modfile.ModulePath(modBytes), nil
golang.org/x/mod/modfile
代码语言:javascript复制func ModuleVersion(path string) (string, error) {
fd := exec.Command("go", "mod", "graph")
然后设置运行路径:
代码语言:javascript复制func KratosMod() string {
cacheOut, _ := exec.Command("go", "env", "GOMODCACHE").Output()
pathOut, _ := exec.Command("go", "env", "GOPATH").Output()
gopath := strings.Trim(string(pathOut), "n")
cachePath = filepath.Join(gopath, "pkg", "mod")
return filepath.Join(gopath, "src", "github.com", "go-kratos", "kratos")
cmd/kratos/internal/base/path.go,默认krotos安装目录是~/.kratos
代码语言:javascript复制func kratosHome() string {
dir, err := os.UserHomeDir()
home := path.Join(dir, ".kratos")
代码语言:javascript复制func Tree(path string, dir string) {
_ = filepath.Walk(path, func(path string, info os.FileInfo, err error) error {
color.GreenString("CREATED")
github.com/fatih/color会对输出进行染色:cmd/kratos/internal/base/repo.go
代码语言:javascript复制type Repo struct {
url string
home string
branch string
}
会调用github API,解析git文件信息,获取当前repo的一些信息,方便后面做diff以及提交代码,本质上是对git 命令进行了一层包装
代码语言:javascript复制func repoDir(url string) string {
vcsURL, err := ParseVCSUrl(url)
代码语言:javascript复制func (r *Repo) Path() string {
func (r *Repo) Pull(ctx context.Context) error {
代码语言:javascript复制func (r *Repo) Clone(ctx context.Context) error {
cmd = exec.CommandContext(ctx, "git", "clone", "-b", r.branch, r.url, r.Path())
cmd/kratos/internal/base/vcs_url.go
代码语言:javascript复制func ParseVCSUrl(repo string) (*url.URL, error) {
if m := scpSyntaxRe.FindStringSubmatch(repo); m != nil {
if !strings.Contains(repo, "//") {
repo = "//" repo
}
if strings.HasPrefix(repo, "//git@") {
repo = "ssh:" repo
} else if strings.HasPrefix(repo, "//") {
repo = "https:" repo
}
repoURL, err = url.Parse(repo)
cmd/kratos/internal/change/change.go
代码语言:javascript复制fmt.Print(ParseCommitsInfo(info))
cmd/kratos/internal/change/get.go
代码语言:javascript复制func (g *GithubAPI) GetReleaseInfo(version string) ReleaseInfo {
func (g *GithubAPI) GetCommitsInfo() []CommitInfo {
info := g.GetReleaseInfo("latest")
url := fmt.Sprintf("https://api.github.com/repos/%s/%s/commits?pre_page=%d&page=%d&since=%s", g.Owner, g.Repo, prePage, page, info.PublishedAt)
代码语言:javascript复制func ParseReleaseInfo(info ReleaseInfo) string {
cmd/kratos/internal/project/add.go
代码语言:javascript复制func (p *Project) Add(ctx context.Context, dir string, layout string, branch string, mod string) error {
e := survey.AskOne(prompt, &override)
A library for building interactive and accessible prompts on terminals supporting ANSI escape sequences.
https://github.com/AlecAivazis/survey提供了命令行交互输入的能力,用户可以根据提示执行对应的操作。我们可以根据交互输入的信息来进行新的repo的生成cmd/kratos/internal/project/new.go
代码语言:javascript复制func (p *Project) New(ctx context.Context, dir string, layout string, branch string) error {
repo := base.NewRepo(layout, branch)
if err := repo.CopyTo(ctx, to, p.Path, []string{".git", ".github"});
cmd/kratos/internal/project/project.go
初始化go mode,然后拉取依赖
代码语言:javascript复制projectRoot := getgomodProjectRoot(wd)
mod, e := base.ModulePath(path.Join(projectRoot, "go.mod"))
代码语言:javascript复制func getgomodProjectRoot(dir string) string {
if dir == filepath.Dir(dir) {
return dir
}
if gomodIsNotExistIn(dir) {
return getgomodProjectRoot(filepath.Dir(dir))
}
return dir
}
cmd/kratos/internal/proto/proto.go,实现了server代码生成和client代码生成的功能
代码语言:javascript复制func init() {
CmdProto.AddCommand(add.CmdAdd)
CmdProto.AddCommand(client.CmdClient)
CmdProto.AddCommand(server.CmdServer)
}
cmd/kratos/internal/proto/add/add.go解析proto文件,根据模板生成最终的golang代码:
代码语言:javascript复制if err := p.Generate(); err != nil {
代码语言:javascript复制func (p *Proto) Generate() error {
body, err := p.execute()
代码语言:javascript复制const protoTemplate = `
syntax = "proto3";
然后是生成service的增删改查命令,本质是对proto插件的一层包装
cmd/kratos/internal/proto/client/client.go
代码语言:javascript复制if err = look("protoc-gen-go", "protoc-gen-go-grpc", "protoc-gen-go-http", "protoc-gen-go-errors", "protoc-gen-openapi"); err != nil {
cmd := exec.Command("kratos", "upgrade")
if strings.HasSuffix(proto, ".proto") {
err = generate(proto, args)
} else {
err = walk(proto, args)
}
代码语言:javascript复制func generate(proto string, args []string) error {
input := []string{
"--proto_path=.",
}
if pathExists(protoPath) {
input = append(input, "--proto_path=" protoPath)
}
inputExt := []string{
"--proto_path=" base.KratosMod(),
"--proto_path=" filepath.Join(base.KratosMod(), "third_party"),
"--go_out=paths=source_relative:.",
"--go-grpc_out=paths=source_relative:.",
"--go-http_out=paths=source_relative:.",
"--go-errors_out=paths=source_relative:.",
"--openapi_out=paths=source_relative:.",
}
cmd/kratos/internal/proto/server/server.go
代码语言:javascript复制proto.Walk(definition,
proto.WithOption(func(o *proto.Option) {
if o.Name == "go_package" {
pkg = strings.Split(o.Constant.Source, ";")[0]
}
}),
proto.WithService(func(s *proto.Service) {
cs := &Service{
Package: pkg,
Service: serviceName(s.Name),
}
for _, e := range s.Elements {
r, ok := e.(*proto.RPC)
if !ok {
continue
}
cs.Methods = append(cs.Methods, &Method{
Service: serviceName(s.Name), Name: serviceName(r.Name), Request: r.RequestType,
Reply: r.ReturnsType, Type: getMethodType(r.StreamsRequest, r.StreamsReturns),
})
}
res = append(res, cs)
}),
)
使用了github.com/emicklei/proto包获得proto的抽象语法树,然后通过访问者模式获得服务信息和接口信息:
代码语言:javascript复制reader, _ := os.Open("test.proto")
parser := proto.NewParser(reader)
definition, _ := parser.Parse()
访问者模式,筛选感兴趣的信息
代码语言:javascript复制proto.Walk(definition,
proto.WithService(handleService),
proto.WithMessage(handleMessage))
b, err := s.execute()
最后渲染模板,生成目标代码
代码语言:javascript复制var serviceTemplate = `
{{- /* delete empty line */ -}}
package service
cmd/protoc-gen-go-errors/main.go实现了proto的自定义字段的解析
代码语言:javascript复制var flags flag.FlagSet
protogen.Options{
ParamFunc: flags.Set,
}.Run(func(gen *protogen.Plugin) error {
代码语言:javascript复制func generateFile(gen *protogen.Plugin, file *protogen.File) *protogen.GeneratedFile {
filename := file.GeneratedFilenamePrefix "_errors.pb.go"
代码语言:javascript复制func generateFileContent(gen *protogen.Plugin, file *protogen.File, g *protogen.GeneratedFile) {
获取错误码和默认错误码,然后通过模板生成对应go代码
代码语言:javascript复制func genErrorsReason(gen *protogen.Plugin, file *protogen.File, g *protogen.GeneratedFile, enum *protogen.Enum) bool {
defaultCode := proto.GetExtension(enum.Desc.Options(), errors.E_DefaultCode)
eCode := proto.GetExtension(v.Desc.Options(), errors.E_Code)
代码语言:javascript复制var errorsTemplate = `
{{ range .Errors }}
cmd/protoc-gen-go-http/main.go,生成http代码是实现了生成http代码的插件google.golang.org/protobuf/compiler/protogen,实现方式类似,最终也是借助模板渲染已经提取的信息
代码语言:javascript复制protogen.Options{
ParamFunc: flag.CommandLine.Set,
}.Run(func(gen *protogen.Plugin) error {
for _, f := range gen.Files {
generateFile(gen, f, *omitempty)
代码语言:javascript复制var httpTemplate = `
{{$svrType := .ServiceType}}
{{$svrName := .ServiceName}}
代码语言:javascript复制func generateFile(gen *protogen.Plugin, file *protogen.File, omitempty bool) *protogen.GeneratedFile {
filename := file.GeneratedFilenamePrefix "_http.pb.go"
g := gen.NewGeneratedFile(filename, file.GoImportPath)
generateFileContent(gen, file, g, omitempty)
代码语言:javascript复制func generateFileContent(gen *protogen.Plugin, file *protogen.File, g *protogen.GeneratedFile, omitempty bool) {
for _, service := range file.Services {
genService(gen, file, g, service, omitempty)
代码语言:javascript复制func genService(gen *protogen.Plugin, file *protogen.File, g *protogen.GeneratedFile, service *protogen.Service, omitempty bool) {
sd.Methods = append(sd.Methods, buildHTTPRule(g, method, rule))
sd.Methods = append(sd.Methods, buildMethodDesc(g, method, http.MethodPost, path))
g.P(sd.execute())
代码语言:javascript复制func buildHTTPRule(g *protogen.GeneratedFile, m *protogen.Method, rule *annotations.HttpRule) *methodDesc {
代码语言:javascript复制func buildMethodDesc(g *protogen.GeneratedFile, m *protogen.Method, method, path string) *methodDesc {
fd := fields.ByName(protoreflect.Name(field))
fields = fd.Message().Fields()
3,config目录
config目录实现了两种情况的配置解析环境变量env和文件file,最核心的功能就是Load加载配置文件和Watch,监听配置内容的变化
代码语言:javascript复制func (f *file) Load() (kvs []*config.KeyValue, err error) {
func (f *file) Watch() (config.Watcher, error) {
文件的解析自然用到了常用的github.com/fsnotify/fsnotify,在外面进行了一层包装,提供统一的接口config.go
代码语言:javascript复制type Config interface {
Load() error
Scan(v interface{}) error
Value(key string) Value
Watch(key string, o Observer) error
Close() error
}