Protocols
- HTTP
- Streaming HTTP (Comet)
- Thrift
- Memcached/Kestrel
- More to come!
Server Support
- Backpressure (to defend against abusive clients)
- Service Registration (e.g., via Zookeeper)
- Native OpenSSL bindings
Client Support
Finagle is a protocol-agnostic, asynchronous RPC system for the JVM that makes it easy to build robust clients and servers in Java, Scala, or any JVM-hosted language.
Rendering even the simplest web page on twitter.com requires the collaboration of dozens of network services speaking many different protocols. For example, in order to render the home page, the application issues requests to the Social Graph Service, Memcached, databases, and many other network services. Each of these speaks a different protocol: Thrift, Memcached, MySQL, and so on. Additionally, many of these services speak to other services -- they are both servers and clients. The Social Graph Service, for instance, provides a Thrift interface but consumes from a cluster of MySQL databases.
In such systems, a frequent cause of outages is poor interaction between components in the presence of failures; common failures include crashed hosts and extreme latency variance. These failures can cascade through the system by causing work queues to back up, TCP connections to churn, or memory and file descriptors to become exhausted. In the worst case, the user sees a Fail Whale.
Finagle provides a robust implementation of:
- connection pools, with throttling to avoid TCP connection churn;
- failure detectors, to identify slow or crashed hosts;
- failover strategies, to direct traffic away from unhealthy hosts;
- load-balancers, including “least-connections” and other strategies; and
- back-pressure techniques, to defend servers against abusive clients and dogpiling.
Additionally, Finagle makes it easier to build and deploy a service that
- publishes standard statistics, logs, and exception reports;
- supports distributed tracing (a la Dapper) across protocols;
- optionally uses ZooKeeper for cluster management; and
- supports common sharding strategies.
Finagle at Twitter
- Changed to a JVM centric approach for hiring and speed of development reasons.
- Scala and Finagle Selection
- Internally they had a lot of people with Ruby and PHP experience, so Scala was appealing.
- Finagle was a compelling factor in choosing Scala. It is a library from Twitter. It handles most of the distributed issues like distributed tracing, service discovery, and service registration. You don’t have to implement all this stuff. It just comes for free.
- Once on the JVM Finagle provided all the primitives they needed (Thrift, ZooKeeper, etc).
- Finagle is being used by Foursquare and Twitter. Scala is also being used by Meetup.
- Like the Thrift application interface. It has really good performance.
- Liked Netty, but wanted out of Java, so Scala was a good choice.
- Picked Finagle because it was cool, knew some of the guys, it worked without a lot of networking code and did all the work needed in a distributed system.
- Node.js wasn’t selected because it is easier to scale the team with a JVM base. Node.js isn’t developed enough to have standards and best practices, a large volume of well tested code. With Scala you can use all the Java code. There’s not a lot of knowledge of how to use it in a scalable way and they target 5ms response times, 4 9s HA, 40K requests per second and some at 400K requests per second. There’s a lot in the Java ecosystem they can leverage.
- Internal services are being shifted from being C/libevent based to being Scala/Finagle based.
- Initially an Actor model was used with Finagle, but that was dropped. For fire and forget work a job queue is used. In addition, Twitter’s utility library contains a Futures implementation and services are implemented in terms of futures. In the situations when a thread pool is needed futures are passed into a future pool. Everything is submitted to the future pool for asynchronous execution.
- Scala encourages no shared state. Finagle is assumed correct because it’s tested by Twitter in production. Mutable state is avoided using constructs in Scala or Finagle. No long running state machines are used. State is pulled from the database, used, and written back to the database. Advantage is developers don’t need to worry about threads or locks.
- (略)
- Redis is used as Gearman’s persistence layer using a memcache proxy built using Finagle.
Finagle 與諸多 Server / Client 框架的主要差別
- 使用 multiplexing io 並讓 asynchronous programming 變的容易
- 內建多種通訊協定，簡化異質網路系統的通訊整合
- 分散式系統架構，可以讓系統容易擴展、容錯與追蹤錯誤
目前己在使用的網路服務
衍生的開放源始碼專案
Client
- memcache / redis (在官方專案)
- cassie - Cassandra Client
- foursquare-fhttp - Http Client(擴充 http 操作)
Server
- Kestrel - message queue(在官方專案)
- finatra - sinatra-like web framework(擴充 http 操作)
Tool
- Scrooge(SBT) - A Thrift generator for Scala(可將 thrift rpc 轉成 finagle service)
不足的地方
- 說明文件不夠完整與學習資源不多
- 缺少應用層面的框架功能
- session / transaction / security management
- [web framework] routing / template ...
- 監控分析工具
distributed tracing
目前提供的官方文件如下表
LEARN MORE
有許多功能的使用方式是目前文件沒寫到的，不足的部份可以在論壇 Finagler 詢問跟爬文，但進階用法還是要讀相關測試案例與原始碼，讀測試案例會比讀 ScalaDoc 更加清楚使用方式
Updated:
Twitter 釋出 distributed tracing 相關的 libraries / servers / web：zipkin
1 comment:
How to register a new account at a casino, and how to find the
There's no signup necessary and you're just 토토 사이트 잘못 환전 샤오 미 downloading 가족 방 the app, and you can get back to play. 토토사이트 제작 벳피스트 Once you've registered, 토토 정류장 you can download the app and 1xbet korea
Post a Comment