Connection Pooling for Connection to Kubernetes #348

New issue

Closed

opened 2026-01-16 12:36:01 +00:00 by tobru · 0 comments

tobru commented

2026-01-16 12:36:01 +00:00

Owner

Stories

As a user, I want to have fast responses when navigating to pages with Kubernetes API calls

Implementation Notes

We currently don't pool connections to the Kubernetes API, which slows down API requests quite a lot. Implement connection pooling to reuse connections for subsequent Kubernetes API calls.

Suggestion from Claud Code:

  Proposed Implementation

  # At module level in service.py (or a new utils file)
  import threading
  from weakref import WeakValueDictionary

  # Thread-safe client pool: {control_plane_pk: ApiClient}
  _kubernetes_client_pool: dict[int, kubernetes.client.ApiClient] = {}
  _pool_lock = threading.Lock()


  class ControlPlane(ServalaModelMixin, models.Model):
      # ... existing fields ...

      def get_kubernetes_client(self):
          """
          Get or create a cached Kubernetes API client.

          Clients are cached per ControlPlane to enable connection pooling.
          The underlying urllib3 PoolManager reuses TCP/TLS connections.
          """
          with _pool_lock:
              # Check if we have a cached client
              if self.pk in _kubernetes_client_pool:
                  return _kubernetes_client_pool[self.pk]

              # Create new client
              api_client = kubernetes.client.ApiClient(self.kubernetes_config)
              if self.proxy_url and self.proxy_url.startswith(("socks4", "socks5")):
                  api_client.rest_client = SOCKSRESTClientObject(
                      self.kubernetes_config, self.proxy_url
                  )

              _kubernetes_client_pool[self.pk] = api_client
              return api_client

      def invalidate_kubernetes_client(self):
          """Remove cached client (call when credentials change)."""
          with _pool_lock:
              _kubernetes_client_pool.pop(self.pk, None)
          # Also clear the cached_property
          self.__dict__.pop('custom_objects_api', None)

      def save(self, *args, **kwargs):
          # Invalidate cached client when credentials might have changed
          if self.pk:
              self.invalidate_kubernetes_client()
          super().save(*args, **kwargs)

  Key Benefits
  ┌──────────────────────────────────────────┬─────────────────────────────────┐
  │                  Before                  │              After              │
  ├──────────────────────────────────────────┼─────────────────────────────────┤
  │ New TLS handshake per page load (~150ms) │ TLS reused across requests      │
  ├──────────────────────────────────────────┼─────────────────────────────────┤
  │ 3 K8s calls = 3 separate connections     │ 3 K8s calls reuse 1 connection  │
  ├──────────────────────────────────────────┼─────────────────────────────────┤
  │ ~320ms per page                          │ Potentially ~150-200ms per page │
  └──────────────────────────────────────────┴─────────────────────────────────┘
  Considerations

  1. Cache invalidation: Call invalidate_kubernetes_client() when credentials change (handled in save())
  2. Memory: One ApiClient per ControlPlane per worker process (minimal)
  3. Stale connections: urllib3 handles connection health internally
  4. Thread safety: Lock protects the dict, urllib3 PoolManager is thread-safe

  Alternative: Even Simpler

  If you want minimal changes, just make get_kubernetes_client a @cached_property:

  @cached_property
  def kubernetes_client(self):
      api_client = kubernetes.client.ApiClient(self.kubernetes_config)
      if self.proxy_url and self.proxy_url.startswith(("socks4", "socks5")):
          api_client.rest_client = SOCKSRESTClientObject(
              self.kubernetes_config, self.proxy_url
          )
      return api_client

  But this only helps within a single request (same ControlPlane model instance). The module-level pool helps across requests.

## Stories _As a user, I want to have fast responses when navigating to pages with Kubernetes API calls_ ## Implementation Notes We currently don't pool connections to the Kubernetes API, which slows down API requests quite a lot. Implement connection pooling to reuse connections for subsequent Kubernetes API calls. Suggestion from Claud Code: ``` Proposed Implementation # At module level in service.py (or a new utils file) import threading from weakref import WeakValueDictionary # Thread-safe client pool: {control_plane_pk: ApiClient} _kubernetes_client_pool: dict[int, kubernetes.client.ApiClient] = {} _pool_lock = threading.Lock() class ControlPlane(ServalaModelMixin, models.Model): # ... existing fields ... def get_kubernetes_client(self): """ Get or create a cached Kubernetes API client. Clients are cached per ControlPlane to enable connection pooling. The underlying urllib3 PoolManager reuses TCP/TLS connections. """ with _pool_lock: # Check if we have a cached client if self.pk in _kubernetes_client_pool: return _kubernetes_client_pool[self.pk] # Create new client api_client = kubernetes.client.ApiClient(self.kubernetes_config) if self.proxy_url and self.proxy_url.startswith(("socks4", "socks5")): api_client.rest_client = SOCKSRESTClientObject( self.kubernetes_config, self.proxy_url ) _kubernetes_client_pool[self.pk] = api_client return api_client def invalidate_kubernetes_client(self): """Remove cached client (call when credentials change).""" with _pool_lock: _kubernetes_client_pool.pop(self.pk, None) # Also clear the cached_property self.__dict__.pop('custom_objects_api', None) def save(self, *args, **kwargs): # Invalidate cached client when credentials might have changed if self.pk: self.invalidate_kubernetes_client() super().save(*args, **kwargs) Key Benefits ┌──────────────────────────────────────────┬─────────────────────────────────┐ │ Before │ After │ ├──────────────────────────────────────────┼─────────────────────────────────┤ │ New TLS handshake per page load (~150ms) │ TLS reused across requests │ ├──────────────────────────────────────────┼─────────────────────────────────┤ │ 3 K8s calls = 3 separate connections │ 3 K8s calls reuse 1 connection │ ├──────────────────────────────────────────┼─────────────────────────────────┤ │ ~320ms per page │ Potentially ~150-200ms per page │ └──────────────────────────────────────────┴─────────────────────────────────┘ Considerations 1. Cache invalidation: Call invalidate_kubernetes_client() when credentials change (handled in save()) 2. Memory: One ApiClient per ControlPlane per worker process (minimal) 3. Stale connections: urllib3 handles connection health internally 4. Thread safety: Lock protects the dict, urllib3 PoolManager is thread-safe Alternative: Even Simpler If you want minimal changes, just make get_kubernetes_client a @cached_property: @cached_property def kubernetes_client(self): api_client = kubernetes.client.ApiClient(self.kubernetes_config) if self.proxy_url and self.proxy_url.startswith(("socks4", "socks5")): api_client.rest_client = SOCKSRESTClientObject( self.kubernetes_config, self.proxy_url ) return api_client But this only helps within a single request (same ControlPlane model instance). The module-level pool helps across requests. ```