SONiC syncd组件

syncd 通过注册回调函数与driver通信,syncd和sai共享命名空间。syncd 启动线程监听共享队列,处理driver的通知。

syncd 进程是介于orchagent与driver之间的进程。syncd从asic-db中读取的数据经转换后调用驱动提供的sai接口下发到硬件,同时需要将驱动的应答进行一定的处理,还需要处理驱动的事件通知(比如端口up/down,mac老化等信息)。处理的消息如下图所示:

@startumlOrchagent -> asic_db : 步骤1asic_db -> syncd : 步骤2syncd -> driver : 步骤3driver -> syncd : 步骤4syncd -> asic_db : 步骤5asic_db -> Orchagent : 步骤6driver -> syncd : 步骤7syncd -> asic_db : 步骤8asic_db -> Orchagent : 步骤9@enduml

S23164.png

orchagent 写操作

create,remove,set写操作:异步写。orchagent会在 sairedis 层构建一个虚拟的sai层:sairedis。orchagent执行sai接口只是对asic-db进行操作,生成或者删除虚拟对象(vid)。默认所有操作都是成功的,直接返回,不等待syncd的应答。执行上图的1和6。syncd从asic-db中读出请求执行上图的2,3,4。如果4步骤返回成功,则整个请求运行结束,否则,syncd将会发送shutdown通知给orchagent。orchagent会退出,如上图的5,6;

orchagent 读操作

get 读操作:同步读。orchagent执行1后会使用select阻塞等待syncd的应答,如果syncd在60分钟内没有应答,那么orchagent会产生segment退出。get操作执行顺序为1->2->3->4->5->6。

// sonic-sairedis/lib/src/sai_redis_generic_get.cpp:257:sai_status_t redis_generic_get(sai_status_t redis_generic_get() {// ...// internal_redis_generic_get    std::string str_object_type = sai_serialize_object_type(object_type);    std::string key = str_object_type + ":" + serialized_object_id;    // 写入本次get事件,不会写输入到asic数据库,只是加入到队列中    g_asicState->set(key, entry, "get");    // 创建临时 select,添加事件,等待响应    swss::Select s;    s.addSelectable(g_redisGetConsumer.get());    //循环等待syncd的应答    while (true)    {        swss::Selectable *sel;        //阻塞等待,时间为GET_RESPONSE_TIMEOUT          int result = s.select(&sel, GET_RESPONSE_TIMEOUT);        //只处理应答情况OBJECT        if (result == swss::Select::OBJECT) {            swss::KeyOpFieldsValuesTuple kco;            g_redisGetConsumer->pop(kco);            const std::string &op = kfvOp(kco);            const std::string &opkey = kfvKey(kco);            if (op != "getresponse") // ignore non response messages            {                continue;            }            sai_status_t status = internal_redis_get_process(                    object_type,                    attr_count,                    attr_list,                    kco);            return status;        }        break;    }    //超时和异常都返回SAI_STATUS_FAILURE    return SAI_STATUS_FAILURE;}

对于get操作,当syncd比较忙的时候,极端情况下会导致orchagent异常退出。

获得driver通知

driver的notify: 驱动检测到硬件事件后,调用syncd注册的回调函数通知syncd,将相关事件写入队列。syncd中有一个专门处理driver-notify的线程ntf-thread。ntf-thread解析driver的notify,然后通过asic-db通知orchagent (orchagent会在主进程的select中监听asic-db。)。执行顺序7->8->9。

注: orchagent 与syncd关于sai这一层非常相似。它们会调用大量的同名函数。这些函数只是名字相同,orchagent调用的是sai-redis库中的函数,而syncd调用的是driver提供的sai库

  • syncd向驱动注册回调函数,syncd定义了几个notify全局函数指针
sai_switch_state_change_notification_fn     on_switch_state_change_ntf = on_switch_state_change;sai_switch_shutdown_request_notification_fn on_switch_shutdown_request_ntf = on_switch_shutdown_request;sai_fdb_event_notification_fn               on_fdb_event_ntf = on_fdb_event;sai_port_state_change_notification_fn       on_port_state_change_ntf = on_port_state_change;sai_packet_event_notification_fn            on_packet_event_ntf = on_packet_event;sai_queue_pfc_deadlock_notification_fn      on_queue_deadlock_ntf = on_queue_deadlock;

syncd和sai共享命名空间,所以驱动直接使用这些函数指针即可调用对应的函数,在初始化的时候将这些全局函数指针通过驱动提供的sai_set_switch_attribute函数设置到sai层。

/** Routine Description:*    Set switch attribute value** Arguments: *   [in] switch_id Switch id*    [in] attr - switch attribute** Return Values:*    SAI_STATUS_SUCCESS on success*    Failure status code on error*/sai_status_t sai_set_switch_attribute(_In_ sai_object_id_t switch_id,                                      _In_ const sai_attribute_t *attr) {  sai_status_t status = SAI_STATUS_SUCCESS;  switch_status_t switch_status = SWITCH_STATUS_SUCCESS;  switch_uint64_t flags = 0;  switch_api_device_info_t api_device_info;  sai_packet_action_t sai_packet_action;  switch_acl_action_t switch_packet_action;  switch_packet_type_t switch_packet_type = SWITCH_PACKET_TYPE_UNICAST;  bool cut_through = false;  if (!attr) {    status = SAI_STATUS_INVALID_PARAMETER;    SAI_LOG_ERROR("null attribute: %s", sai_status_to_string(status));    return status;  }  memset(&api_device_info, 0x0, sizeof(api_device_info));  if (status != SAI_STATUS_SUCCESS) {    return status;  }  if (attr->id <= SAI_SWITCH_ATTR_ACL_STAGE_EGRESS) {  // Unsupported    SAI_LOG_DEBUG("Switch attribute set: %s", switch_attr_name[attr->id]);  }  switch (attr->id) {    //......    case SAI_SWITCH_ATTR_FDB_EVENT_NOTIFY:      sai_switch_notifications.on_fdb_event = attr->value.ptr;      break;    case SAI_SWITCH_ATTR_PORT_STATE_CHANGE_NOTIFY:      sai_switch_notifications.on_port_state_change = attr->value.ptr;      break;    case SAI_SWITCH_ATTR_PACKET_EVENT_NOTIFY:      sai_switch_notifications.on_packet_event = attr->value.ptr;      break;    case SAI_SWITCH_ATTR_SWITCH_STATE_CHANGE_NOTIFY:      sai_switch_notifications.on_switch_state_change = attr->value.ptr;      break;    case SAI_SWITCH_ATTR_SHUTDOWN_REQUEST_NOTIFY:      sai_switch_notifications.on_switch_shutdown_request = attr->value.ptr;      break;    ......    default:      SAI_LOG_ERROR("Unsupported Switch attribute: %d", attr->id);      // Unsupported: Temporary hack till all attrs are supported      switch_status = SWITCH_STATUS_SUCCESS;  }  //......}

sai接口初始化的时候会向驱动注册回调函数,回调函数中会调用我们注册的全局函数指针,我们以fdb为例进行说明:

sai_status_t sai_fdb_initialize(sai_api_service_t *sai_api_service) {  sai_api_service->fdb_api = fdb_api;  switch_uint16_t mac_event_flags = 0;  mac_event_flags |= SWITCH_MAC_EVENT_LEARN | SWITCH_MAC_EVENT_AGE |                     SWITCH_MAC_EVENT_MOVE | SWITCH_MAC_EVENT_DELETE;  //初始化fdb的sai接口的时候,向驱动注册了 sai_mac_notify_cb 回调函数。  switch_api_mac_notification_register(      device, SWITCH_SAI_APP_ID, mac_event_flags, &sai_mac_notify_cb);  switch_api_mac_table_set_learning_timeout(device, SAI_L2_LEARN_TIMEOUT);  return SAI_STATUS_SUCCESS;}static void sai_mac_notify_cb(const switch_device_t device,                              const uint16_t num_entries,                              const switch_api_mac_entry_t *mac_entry,                              const switch_mac_event_t mac_event,                              void *app_data) {  SAI_LOG_ENTER();  sai_fdb_event_notification_data_t fdb_event[num_entries];  sai_attribute_t attr_lists[num_entries][2];  uint16_t entry = 0;  for (entry = 0; entry < num_entries; entry++) {    memset(&fdb_event[entry], 0, sizeof(fdb_event[entry]));    fdb_event[entry].event_type = switch_mac_event_to_sai_fdb_event(mac_event);    memcpy(fdb_event[entry].fdb_entry.mac_address,           mac_entry[entry].mac.mac_addr,           ETH_ALEN);    fdb_event[entry].fdb_entry.switch_id =        (((unsigned long)SWITCH_HANDLE_TYPE_DEVICE)         << SWITCH_HANDLE_TYPE_SHIFT) |        0x1;    fdb_event[entry].fdb_entry.bv_id = mac_entry[entry].network_handle;    memset(attr_lists[entry], 0, sizeof(attr_lists[entry]));    attr_lists[entry][0].id = SAI_FDB_ENTRY_ATTR_TYPE;    attr_lists[entry][0].value.s32 = SAI_FDB_ENTRY_TYPE_DYNAMIC;    attr_lists[entry][1].id = SAI_FDB_ENTRY_ATTR_BRIDGE_PORT_ID;    attr_lists[entry][1].value.oid = mac_entry->handle;    fdb_event[entry].attr_count = 2;    if (fdb_event[entry].event_type == SAI_FDB_EVENT_FLUSHED) {      // Overwriting now for SONiC to be able to process it correctly      fdb_event[entry].event_type = SAI_FDB_EVENT_AGED;    }    fdb_event[entry].attr = attr_lists[entry];  }  //调用syncd的回调函数  sai_switch_notifications.on_fdb_event(num_entries, fdb_event);  return;}

syncd 启动 notify 线程

std::shared_ptr ntf_process_thread;void startNotificationsProcessingThread(){    runThread = true;    ntf_process_thread = std::make_shared(ntf_process_function);}void ntf_process_function(){    while (runThread) {        cv.wait(ulock);        // this is notifications processing thread context, which is different        // from SAI notifications context, we can safe use g_mutex here,        // processing each notification is under same mutex as processing main        // events, counters and reinit        swss::KeyOpFieldsValuesTuple item;        while (tryDequeue(item))//从队列中取出notify        {            processNotification(item);//处理notify        }    }}bool tryDequeue(        _Out_ swss::KeyOpFieldsValuesTuple &item){    std::lock_guard lock(queue_mutex);    if (ntf_queue.empty()) {        return false;    }    item = ntf_queue.front();    ntf_queue.pop();    return true;}void processNotification (        _In_ const swss::KeyOpFieldsValuesTuple &item){    std::lock_guard lock(g_mutex);    SWSS_LOG_ENTER();    std::string notification = kfvKey(item);    std::string data = kfvOp(item);    if (notification == "switch_state_change") {        handle_switch_state_change(data);    } else if (notification == "fdb_event") {        handle_fdb_event(data);    }    else if (notification == "port_state_change")    {        handle_port_state_change(data);    }    else if (notification == "switch_shutdown_request")    {        handle_switch_shutdown_request(data);    }    else if (notification == "queue_deadlock")    {        handle_queue_deadlock(data);    }    else    {        SWSS_LOG_ERROR("unknow notification: %s", notification.c_str());    }}// 以 fdb 为例void handle_fdb_event(        _In_ const std::string &data){    SWSS_LOG_ENTER();    uint32_t count;    sai_fdb_event_notification_data_t *fdbevent = NULL;    sai_deserialize_fdb_event_ntf(data, count, &fdbevent);    process_on_fdb_event(count, fdbevent);    sai_deserialize_free_fdb_event_ntf(count, fdbevent);}void process_on_fdb_event(        _In_ uint32_t count,        _In_ sai_fdb_event_notification_data_t *data){    for (uint32_t i = 0; i < count; i++) {        sai_fdb_event_notification_data_t *fdb = &data[i];        fdb->fdb_entry.switch_id = translate_rid_to_vid(fdb->fdb_entry.switch_id, SAI_NULL_OBJECT_ID);        fdb->fdb_entry.bv_id = translate_rid_to_vid(fdb->fdb_entry.bv_id, fdb->fdb_entry.switch_id);        translate_rid_to_vid_list(SAI_OBJECT_TYPE_FDB_ENTRY, fdb->fdb_entry.switch_id, fdb->attr_count, fdb->attr);        /*         * Currently because of bcrm bug, we need to install fdb entries in         * asic view and currently this event don't have fdb type which is         * required on creation.         */        redisPutFdbEntryToAsicView(fdb);    }    std::string s = sai_serialize_fdb_event_ntf(count, data);    send_notification("fdb_event", s);}void send_notification(        _In_ std::string op,        _In_ std::string data,        _In_ std::vector &entry){    //写入数据库    notifications->send(op, data, entry);}void send_notification(        _In_ std::string op,        _In_ std::string data){    SWSS_LOG_ENTER();    std::vector entry;    send_notification(op, data, entry);}

syncd与SDE交互

image.png

在SONiC中,syncd容器是与ASIC通信的唯一通道,控制面的业务进程想将数据下发ASIC时,最终处理流程都是将数据按照SAI标准接口格式写入Redis中的ASIC_DB,syncd容器中的同名主进程syncd订阅ASIC_DB中的相关表项并处理这些下发的数据,调用ASIC SDK对SAI API的实现,并通过ASIC driver下发到ASIC。

其中SDE就包括了ASIC SDK以及运行在Tofino ASIC上的tofino.bin(由P4源码编译生成的二进制程序)。

nfvschool 微信公共号: nfvschool
nfvschool网址: nfvschool.cn

Was this helpful?

2 / 0

发表回复 0