重写Nacos服务发现:多个服务器如何跨命名空间,访问公共服务?
一、问题背景
在开发某个公共应用时,笔者发现该公共应用的数据是所有测试环境(假设存在 dev/dev2/dev3)通用的。
这就意味着只需部署一个应用,就能满足所有测试环境的需求;也意味着所有测试环境都需要调用该公共应用,而不同测试环境的应用注册在不同的 Nacos 命名空间。
二、两种解决方案
如果所有测试环境都需要调用该公共应用,有两种可行的方案。第一种,将该公共服务同时注册到不同的测试环境所对应的命名空间中。
第二种,将公共应用注册到单独的命名空间,不同的测试环境能够跨命名空间访问该应用。
三、详细的问题解决过程
先行交代笔者的版本号配置。Nacos 客户端版本号为 NACOS 1.4.1
;Java 项目的 Nacos 版本号如下。
最初想法是将该公共应用同时注册到多个命名空间下。
01 注册多个命名空间
从该博客中,我们看到其他程序员朋友也遇到了类似的公共服务的需求。在本篇文章中,笔者将进一步分享实现思路以及示例代码。
说明:以下代码内容来自用户 chuntaojun 的分享。
shareNamespace={namespaceId[:group]},{namespaceId[:group]} 复制代码
@RunWith(SpringRunner.class)@SpringBootTest(classes = NamingApp.class, properties = {"server.servlet.context-path=/nacos"}, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)public class SelectServiceInShareNamespace_ITCase { private NamingService naming1; private NamingService naming2; @LocalServerPort private int port; @Before public void init() throws Exception{ NamingBase.prepareServer(port); if (naming1 == null) { Properties properties = new Properties(); properties.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port); properties.setProperty(PropertyKeyConst.SHARE_NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36"); naming1 = NamingFactory.createNamingService(properties); Properties properties2 = new Properties(); properties2.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port); properties2.setProperty(PropertyKeyConst.NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36"); naming2 = NamingFactory.createNamingService(properties2); } while (true) { if (!"UP".equals(naming1.getServerStatus())) { Thread.sleep(1000L); continue; } break; } } @Test public void testSelectInstanceInShareNamespaceNoGroup() throws NacosException, InterruptedException { String service1 = randomDomainName(); String service2 = randomDomainName(); naming1.registerInstance(service1, "127.0.0.1", 90); naming2.registerInstance(service2, "127.0.0.2", 90); Thread.sleep(1000); List instances = naming1.getAllInstances(service2); Assert.assertEquals(1, instances.size()); Assert.assertEquals(service2, NamingUtils.getServiceName(instances.get(0).getServiceName())); } @Test public void testSelectInstanceInShareNamespaceWithGroup() throws NacosException, InterruptedException { String service1 = randomDomainName(); String service2 = randomDomainName(); naming2.registerInstance(service1, groupName, "127.0.0.1", 90); naming3.registerInstance(service2, "127.0.0.2", 90); Thread.sleep(1000); List instances = naming3.getAllInstances(service1); Assert.assertEquals(1, instances.size()); Assert.assertEquals(service1, NamingUtils.getServiceName(instances.get(0).getServiceName())); Assert.assertEquals(groupName, NamingUtils.getServiceName(NamingUtils.getGroupName(instances.get(0).getServiceName()))); }}复制代码
进一步考虑后发现该解决方案可能不太契合当前遇到的问题。公司目前的开发测试环境有很多个,并且不确定以后会不会继续增加。
如果每增加一个环境,都需要修改一次公共服务的配置,并且重启一次公共服务,着实太麻烦了。倒不如反其道而行,让其他的服务器实现跨命名空间访问公共服务。
02 跨命名空间访问
针对实际问题查找资料时,我们找到了类似的参考分享《重写 Nacos 服务发现逻辑动态修改远程服务IP地址》。
跟着博客思路看代码,笔者了解到服务发现的主要相关类是 NacosNamingService
, NacosDiscoveryProperties
, NacosDiscoveryAutoConfiguration
。
然后,笔者将博客的示例代码复制过来,试着进行如下调试:
@Slf4j@Configuration@ConditionalOnNacosDiscoveryEnabled@ConditionalOnProperty( name = {"spring.profiles.active"}, havingValue = "dev")@AutoConfigureBefore({NacosDiscoveryClientAutoConfiguration.class})public class DevEnvironmentNacosDiscoveryClient { @Bean @ConditionalOnMissingBean public NacosDiscoveryProperties nacosProperties() { return new DevEnvironmentNacosDiscoveryProperties(); } static class DevEnvironmentNacosDiscoveryProperties extends NacosDiscoveryProperties { private NamingService namingService; @Override public NamingService namingServiceInstance() { if (null != this.namingService) { return this.namingService; } else { Properties properties = new Properties(); properties.put("serverAddr", super.getServerAddr()); properties.put("namespace", super.getNamespace()); properties.put("com.alibaba.nacos.naming.log.filename", super.getLogName()); if (super.getEndpoint().contains(":")) { int index = super.getEndpoint().indexOf(":"); properties.put("endpoint", super.getEndpoint().substring(0, index)); properties.put("endpointPort", super.getEndpoint().substring(index + 1)); } else { properties.put("endpoint", super.getEndpoint()); } properties.put("accessKey", super.getAccessKey()); properties.put("secretKey", super.getSecretKey()); properties.put("clusterName", super.getClusterName()); properties.put("namingLoadCacheAtStart", super.getNamingLoadCacheAtStart()); try { this.namingService = new DevEnvironmentNacosNamingService(properties); } catch (Exception var3) { log.error("create naming service error!properties={},e=,", this, var3); return null; } return this.namingService; } } } static class DevEnvironmentNacosNamingService extends NacosNamingService { public DevEnvironmentNacosNamingService(Properties properties) { super(properties); } @Override public List selectInstances(String serviceName, List clusters, boolean healthy) throws NacosException { List instances = super.selectInstances(serviceName, clusters, healthy); instances.stream().forEach(instance -> instance.setIp("10.101.232.24")); return instances; } }}复制代码
调试后发现博客提供的代码并不能满足笔者的需求,还得进一步深入探索。
但幸运的是,调试过程发现 Nacos 服务发现的关键类是 com.alibaba.cloud.nacos.discovery.NacosServiceDiscovery
,其中的关键方法是 getInstances()
和 getServices()
,即「返回指定服务 ID 的所有服务实例」和「获取所有服务的名称」。
也就是说,对 getInstances()
方法进行重写肯定能实现本次目标——跨命名空间访问公共服务。
public List getInstances(String serviceId) throws NacosException { String group = discoveryProperties.getGroup(); List instances = discoveryProperties.namingServiceInstance() .selectInstances(serviceId, group, true); return hostToServiceInstanceList(instances, serviceId);}public List getServices() throws NacosException { String group = discoveryProperties.getGroup(); ListView services = discoveryProperties.namingServiceInstance() .getServicesOfServer(1, Integer.MAX_VALUE, group); return services.getData();}复制代码
03 最终解决思路及代码示例
具体的解决方案思路大致如下:
-
生成一个共享配置类
NacosShareProperties
,用来配置共享公共服务的namespace
和group
; -
重写配置类
NacosDiscoveryProperties
(新:NacosDiscoveryPropertiesV2
),将新增的共享配置类作为属性放进该配置类,后续会用到; -
重写服务发现类
NacosServiceDiscovery
(新:NacosServiceDiscoveryV2
),这是最关键的逻辑; -
重写自动配置类
NacosDiscoveryAutoConfiguration
,将自定义相关类比 Nacos 原生类更早的注入容器。
最终代码中用到了一些工具类,可以自行补充完整。
@Configuration@ConfigurationProperties(prefix = "nacos.share")public class NacosShareProperties { private final Map> NAMESPACE_TO_GROUP_NAME_MAP = new ConcurrentHashMap<>(); private List entities; public List getEntities() { return entities; } public void setEntities(List entities) { this.entities = entities; } public Map> getNamespaceGroupMap() { safeStream(entities).filter(entity -> nonNull(entity) && nonNull(entity.getNamespace())) .forEach(entity -> { Set groupNames = NAMESPACE_TO_GROUP_NAME_MAP.computeIfAbsent(entity.getNamespace(), k -> new HashSet<>()); if (nonNull(entity.getGroupNames())) groupNames.addAll(entity.getGroupNames()); }); return new HashMap<>(NAMESPACE_TO_GROUP_NAME_MAP); } @Override public String toString() { return "NacosShareProperties{" + "entities=" + entities + '}'; } public static final class NacosShareEntity { private String namespace; private List groupNames; public String getNamespace() { return namespace; } public void setNamespace(String namespace) { this.namespace = namespace; } public List getGroupNames() { return groupNames; } public void setGroupNames(List groupNames) { this.groupNames = groupNames; } @Override public String toString() { return "NacosShareEntity{" + "namespace='" + namespace + ''' + ", groupNames=" + groupNames + '}'; } }}复制代码
public class NacosDiscoveryPropertiesV2 extends NacosDiscoveryProperties { private static final Logger log = LoggerFactory.getLogger(NacosDiscoveryPropertiesV2.class); private final NacosShareProperties nacosShareProperties; private static final Map NAMESPACE_TO_NAMING_SERVICE_MAP = new ConcurrentHashMap<>(); public NacosDiscoveryPropertiesV2(NacosShareProperties nacosShareProperties) { super(); this.nacosShareProperties = nacosShareProperties; } public Map shareNamingServiceInstances() { if (!NAMESPACE_TO_NAMING_SERVICE_MAP.isEmpty()) { return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP); } List entities = Optional.ofNullable(nacosShareProperties) .map(NacosShareProperties::getEntities).orElse(Collections.emptyList()); entities.stream().filter(entity -> nonNull(entity) && nonNull(entity.getNamespace())) .filter(PredicateUtil.distinctByKey(NacosShareProperties.NacosShareEntity::getNamespace)) .forEach(entity -> { try { NamingService namingService = NacosFactory.createNamingService(getNacosProperties(entity.getNamespace())); if (namingService != null) { NAMESPACE_TO_NAMING_SERVICE_MAP.put(entity.getNamespace(), namingService); } } catch (Exception e) { log.error("create naming service error! properties={}, e=", this, e); } }); return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP); } private Properties getNacosProperties(String namespace) { Properties properties = new Properties(); properties.put(SERVER_ADDR, getServerAddr()); properties.put(USERNAME, Objects.toString(getUsername(), "")); properties.put(PASSWORD, Objects.toString(getPassword(), "")); properties.put(NAMESPACE, namespace); properties.put(UtilAndComs.NACOS_NAMING_LOG_NAME, getLogName()); String endpoint = getEndpoint(); if (endpoint.contains(":")) { int index = endpoint.indexOf(":"); properties.put(ENDPOINT, endpoint.substring(0, index)); properties.put(ENDPOINT_PORT, endpoint.substring(index + 1)); } else { properties.put(ENDPOINT, endpoint); } properties.put(ACCESS_KEY, getAccessKey()); properties.put(SECRET_KEY, getSecretKey()); properties.put(CLUSTER_NAME, getClusterName()); properties.put(NAMING_LOAD_CACHE_AT_START, getNamingLoadCacheAtStart());// enrichNacosDiscoveryProperties(properties); return properties; }}复制代码
public class NacosServiceDiscoveryV2 extends NacosServiceDiscovery { private final NacosDiscoveryPropertiesV2 discoveryProperties; private final NacosShareProperties nacosShareProperties; private final NacosServiceManager nacosServiceManager; public NacosServiceDiscoveryV2(NacosDiscoveryPropertiesV2 discoveryProperties, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager) { super(discoveryProperties, nacosServiceManager); this.discoveryProperties = discoveryProperties; this.nacosShareProperties = nacosShareProperties; this.nacosServiceManager = nacosServiceManager; } public List getInstances(String serviceId) throws NacosException { String group = discoveryProperties.getGroup(); List instances = discoveryProperties.namingServiceInstance() .selectInstances(serviceId, group, true); if (isEmpty(instances)) { Map> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap(); Map namespace2NamingServiceMap = discoveryProperties.shareNamingServiceInstances(); for (Map.Entry entry : namespace2NamingServiceMap.entrySet()) { String namespace; NamingService namingService; if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue())) continue; Set groupNames = namespaceGroupMap.get(namespace); List shareInstances; if (isEmpty(groupNames)) { shareInstances = namingService.selectInstances(serviceId, group, true); if (nonEmpty(shareInstances)) break; } else { shareInstances = new ArrayList<>(); for (String groupName : groupNames) { List subShareInstances = namingService.selectInstances(serviceId, groupName, true); if (nonEmpty(subShareInstances)) { shareInstances.addAll(subShareInstances); } } } if (nonEmpty(shareInstances)) { instances = shareInstances; break; } } } return hostToServiceInstanceList(instances, serviceId); } public List getServices() throws NacosException { String group = discoveryProperties.getGroup(); ListView services = discoveryProperties.namingServiceInstance() .getServicesOfServer(1, Integer.MAX_VALUE, group); return services.getData(); } public static List hostToServiceInstanceList( List instances, String serviceId) { List result = new ArrayList<>(instances.size()); for (Instance instance : instances) { ServiceInstance serviceInstance = hostToServiceInstance(instance, serviceId); if (serviceInstance != null) { result.add(serviceInstance); } } return result; } public static ServiceInstance hostToServiceInstance(Instance instance, String serviceId) { if (instance == null || !instance.isEnabled() || !instance.isHealthy()) { return null; } NacosServiceInstance nacosServiceInstance = new NacosServiceInstance(); nacosServiceInstance.setHost(instance.getIp()); nacosServiceInstance.setPort(instance.getPort()); nacosServiceInstance.setServiceId(serviceId); Map metadata = new HashMap<>(); metadata.put("nacos.instanceId", instance.getInstanceId()); metadata.put("nacos.weight", instance.getWeight() + ""); metadata.put("nacos.healthy", instance.isHealthy() + ""); metadata.put("nacos.cluster", instance.getClusterName() + ""); metadata.putAll(instance.getMetadata()); nacosServiceInstance.setMetadata(metadata); if (metadata.containsKey("secure")) { boolean secure = Boolean.parseBoolean(metadata.get("secure")); nacosServiceInstance.setSecure(secure); } return nacosServiceInstance; } private NamingService namingService() { return nacosServiceManager .getNamingService(discoveryProperties.getNacosProperties()); }}复制代码
@Configuration(proxyBeanMethods = false)@ConditionalOnDiscoveryEnabled@ConditionalOnNacosDiscoveryEnabled@AutoConfigureBefore({NacosDiscoveryAutoConfiguration.class})public class NacosDiscoveryAutoConfigurationV2 { @Bean @ConditionalOnMissingBean public NacosDiscoveryPropertiesV2 nacosProperties(NacosShareProperties nacosShareProperties) { return new NacosDiscoveryPropertiesV2(nacosShareProperties); } @Bean @ConditionalOnMissingBean public NacosServiceDiscovery nacosServiceDiscovery( NacosDiscoveryPropertiesV2 discoveryPropertiesV2, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager ) { return new NacosServiceDiscoveryV2(discoveryPropertiesV2, nacosShareProperties, nacosServiceManager); }}复制代码
本以为问题到这就结束了,但最后自测时发现程序根本不走 Nacos
的服务发现逻辑,而是执行 Ribbon
的负载均衡逻辑com.netflix.loadbalancer.AbstractLoadBalancerRule
。
不过实现类是 com.alibaba.cloud.nacos.ribbon.NacosRule
,继续基于 NacosRule
重写负载均衡。
public class ShareNacosNamespaceRule extends AbstractLoadBalancerRule { private static final Logger LOGGER = LoggerFactory.getLogger(ShareNacosNamespaceRule.class); @Autowired private NacosDiscoveryPropertiesV2 nacosDiscoveryPropertiesV2; @Autowired private NacosShareProperties nacosShareProperties; @SneakyThrows @Override public Server choose(Object key) { try { String clusterName = this.nacosDiscoveryPropertiesV2.getClusterName(); DynamicServerListLoadBalancer loadBalancer = (DynamicServerListLoadBalancer) getLoadBalancer(); String name = loadBalancer.getName(); NamingService namingService = nacosDiscoveryPropertiesV2 .namingServiceInstance(); List instances = namingService.selectInstances(name, true); if (CollectionUtils.isEmpty(instances)) { LOGGER.warn("no instance in service {}, then to get share service's instance", name); List shareNamingService = this.getShareNamingService(name); if (nonEmpty(shareNamingService)) instances = shareNamingService; else return null; } List instancesToChoose = instances; if (org.apache.commons.lang3.StringUtils.isNotBlank(clusterName)) { List sameClusterInstances = instances.stream() .filter(instance -> Objects.equals(clusterName, instance.getClusterName())) .collect(Collectors.toList()); if (!CollectionUtils.isEmpty(sameClusterInstances)) { instancesToChoose = sameClusterInstances; } else { LOGGER.warn( "A cross-cluster call occurs,name = {}, clusterName = {}, instance = {}", name, clusterName, instances); } } Instance instance = ExtendBalancer.getHostByRandomWeight2(instancesToChoose); return new NacosServer(instance); } catch (Exception e) { LOGGER.warn("NacosRule error", e); return null; } } @Override public void initWithNiwsConfig(IClientConfig iClientConfig) { } private List getShareNamingService(String serviceId) throws NacosException { List instances = Collections.emptyList(); Map> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap(); Map namespace2NamingServiceMap = nacosDiscoveryPropertiesV2.shareNamingServiceInstances(); for (Map.Entry entry : namespace2NamingServiceMap.entrySet()) { String namespace; NamingService namingService; if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue())) continue; Set groupNames = namespaceGroupMap.get(namespace); List shareInstances; if (isEmpty(groupNames)) { shareInstances = namingService.selectInstances(serviceId, true); if (nonEmpty(shareInstances)) break; } else { shareInstances = new ArrayList<>(); for (String groupName : groupNames) { List subShareInstances = namingService.selectInstances(serviceId, groupName, true); if (nonEmpty(subShareInstances)) { shareInstances.addAll(subShareInstances); } } } if (nonEmpty(shareInstances)) { instances = shareInstances; break; } } return instances; }}复制代码
至此问题得以解决。
在 Nacos
上配置好共享 namespace
和 group
后,就能够进行跨命名空间访问了。
# nacos共享命名空间配置 示例nacos.share.entities[0].namespace=e6ed2017-3ed6-4d9b-824a-db626424fc7bnacos.share.entities[0].groupNames[0]=DEFAULT_GROUP# 指定服务使用共享的负载均衡规则,service-id是注册到nacos上的服务id,ShareNacosNamespaceRule需要写全限定名service-id.ribbon.NFLoadBalancerRuleClassName=***.***.***.ShareNacosNamespaceRule复制代码
注意:如果 Java 项目的 nacos discovery
版本用的是 2021.1
,则不需要重写 Ribbon 的负载均衡类,因为该版本的 Nacos 不依赖 Ribbon。
2.2.1.RELEASE 版本的 nacos discovery
依赖 Ribbon.
2021.1 版本的 nacos discovery
不依赖 Ribbon。
四、总结
为了达到共享命名空间的预期,构思、查找资料、实现逻辑、调试,前后一共花费 4 天左右。
但该功能仍然存在共享服务缓存等可优化空间,留待后续实现。
来源地址:https://blog.csdn.net/m0_71777195/article/details/128350690
免责声明:
① 本站未注明“稿件来源”的信息均来自网络整理。其文字、图片和音视频稿件的所属权归原作者所有。本站收集整理出于非商业性的教育和科研之目的,并不意味着本站赞同其观点或证实其内容的真实性。仅作为临时的测试数据,供内部测试之用。本站并未授权任何人以任何方式主动获取本站任何信息。
② 本站未注明“稿件来源”的临时测试数据将在测试完成后最终做删除处理。有问题或投稿请发送至: 邮箱/279061341@qq.com QQ/279061341