packages/rdma-core
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Compare commits

base: packages:9e7e7ff53904dc8ce156fadde787c53840f1116e
Branches Tags
packages:main
...
compare: packages:2772898e6e73cb4b1353af3dcf364a579290b151
Branches Tags
packages:main

11 Commits
9e7e7ff539 ... 2772898e6e

Author SHA1 Message Date
openeuler-ci-bot
2772898e6e
!243 [sync] PR-242: libxscale: update to version 2412GA 
From: @openeuler-sync-bot 
Reviewed-by: @hginjgerx 
Signed-off-by: @hginjgerx
 2025-05-14 08:10:05 +00:00
Xin Tian
0358e09981 libxscale: update to version 2412GA 
new feature:
- support diamond products
- support ibv_wr apis
- support extended CQ poll apis

bugfix:
- imm data endian error

Signed-off-by: Xin Tian <tianx@yunsilicon.com>
(cherry picked from commit d12a87881cb9254bb46cfdde6131428a139f15bb)
 2025-05-09 15:10:33 +08:00
openeuler-ci-bot
b418244a4d
!240 [sync] PR-237: libhns: Bugfixes and one debug improvement 
From: @openeuler-sync-bot 
Reviewed-by: @hginjgerx 
Signed-off-by: @hginjgerx
 2025-04-29 05:51:29 +00:00
Xinghai Cen
1f0e4e17fd libhns: Bugfixes and one debug improvement 
The last commit was found when I created a XRC SRQ in
lock-free mode but failed to destroy it because of the
refcnt check added in the previous commit.

The failure was because the PAD was acquired through
ibv_srq->pd in destroy_srq(), while ibv_srq->pd wasn't
assigned when the SRQ was created by ibv_create_srq_ex().
So let's assign ibv_srq->pd in the common ibv_icmd_create_srq() ,
so that drivers can get the correct pd no matter
which api the SRQ is created by.

Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
(cherry picked from commit 3ac30fc125c7cff122f21ff8593294060c92429f)
 2025-04-29 09:55:35 +08:00
openeuler-ci-bot
d43abea599
!234 [sync] PR-231: libhns: Add support for LTTng tracing 
From: @openeuler-sync-bot 
Reviewed-by: @hginjgerx 
Signed-off-by: @hginjgerx
 2025-04-25 08:20:03 +00:00
Xinghai Cen
9c294146d7 libhns: Add support for LTTng tracing 
Add support for LTTng tracing. For now it is used for post_send, post_recv and poll_cq.

Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
(cherry picked from commit 76d4645e3b8a9e57b723e3f7297bda62e934d6f2)
 2025-04-25 15:53:09 +08:00
openeuler-ci-bot
b8c8fcbcd2
!226 Sync some patches of master 
From: @cxh269 
Reviewed-by: @hginjgerx 
Signed-off-by: @hginjgerx
 2025-04-23 07:36:14 +00:00
Xinghai Cen
703ae1270e libhns: Cleanup and Bugfixes 
Cleanup and Bugfixes:
        0053-libhns-Clean-up-data-type-issues.patch
        0054-libhns-Fix-wrong-max-inline-data-value.patch
        0055-libhns-Fix-wrong-order-of-spin-unlock-in-modify-qp.patch

Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
 2025-04-23 10:30:21 +08:00
Xinghai Cen
5a10c8bdbf fix some libzrdma bugs and add some optimization 
Signed-off-by: 李富艳 <li.fuyan@zte.com.cn>
 2025-04-23 10:30:13 +08:00
openeuler-ci-bot
e2c25aebb4
!219 libhns: Fixes some bugs for libhns 
From: @cxh269 
Reviewed-by: @hginjgerx 
Signed-off-by: @hginjgerx
 2025-03-31 12:54:20 +00:00
Xinghai Cen
4bfa460dd7 libhns: Fixes some bugs for libhns 
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IB66RT

------------------------------------------------------------------

Changes to be committed:
      modified:   0040-libhns-Fix-memory-leakage-when-DCA-is-enabled.patch
      new file:   0045-libhns-fix-incorrectly-using-fixed-pagesize.patch
      new file:   0046-libhns-fix-missing-new-IO-support-for-DCA.patch

Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
 2025-03-31 17:07:54 +08:00
20 changed files with 6143 additions and 9 deletions
Show Stats Expand all files Collapse all files

15
0040-libhns-Fix-memory-leakage-when-DCA-is-enabled.patch
View File

@ -1,4 +1,4 @@
From af20dd32df73ff72d35a430a5fb87ac42d70cdf4 Mon Sep 17 00:00:00 2001
From fa9d2bc71a0a524e82d15023fcdfe9b9e3931a24 Mon Sep 17 00:00:00 2001
From: wenglianfa <wenglianfa@huawei.com>
Date: Thu, 25 Jul 2024 11:06:01 +0800
Subject: [PATCH] libhns: Fix memory leakage when DCA is enabled
@ -17,23 +17,26 @@ Fixes: 2783884a97e7 ("libhns: Add support for attaching QP's WQE buffer")
Signed-off-by: wenglianfa <wenglianfa@huawei.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u_verbs.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
providers/hns/hns_roce_u_verbs.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index e30880c..c733b21 100644
index e30880c..bfa25ff 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -1357,7 +1357,8 @@ static void qp_free_wqe(struct hns_roce_qp *qp)
@@ -1357,7 +1357,11 @@ static void qp_free_wqe(struct hns_roce_qp *qp)
if (qp->rq.wqe_cnt)
free(qp->rq.wrid);
- hns_roce_free_buf(&qp->buf);
+
+ if (qp->dca_wqe.bufs)
+ free(qp->dca_wqe.bufs);
+ else
+ hns_roce_free_buf(&qp->buf);
}
static int qp_alloc_wqe(struct ibv_qp_init_attr_ex *attr,
--
2.33.0
2.25.1

110
0046-libhns-fix-incorrectly-using-fixed-pagesize.patch Normal file
View File

@ -0,0 +1,110 @@
From 7bd22fed52a1828b0d44a990b52266e9e1d92b5d Mon Sep 17 00:00:00 2001
From: Chengchang Tang <tangchengchang@huawei.com>
Date: Tue, 30 Jan 2024 21:00:46 +0800
Subject: [PATCH 45/46] libhns: fix incorrectly using fixed pagesize
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IB66RT
------------------------------------------------------------------
Currently, actually used page size is fixed, causing the flexible wqe
buffer size feature to not take effect.
Fixes: 9ab7600d832b ("libhns: Add support for attaching QP's WQE buffer")
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u_verbs.c | 24 +++++++++++++-----------
1 file changed, 13 insertions(+), 11 deletions(-)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index bce215e..848f836 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -1296,14 +1296,14 @@ static void free_recv_rinl_buf(struct hns_roce_rinl_buf *rinl_buf)
static void get_best_multi_region_pg_shift(struct hns_roce_device *hr_dev,
struct hns_roce_context *ctx,
- struct hns_roce_qp *qp)
+ struct hns_roce_qp *qp, bool dca_en)
{
uint32_t ext_sge_size;
uint32_t sq_size;
uint32_t rq_size;
uint8_t pg_shift;
- if (!(ctx->config & HNS_ROCE_UCTX_RSP_DYN_QP_PGSZ)) {
+ if (!(ctx->config & HNS_ROCE_UCTX_RSP_DYN_QP_PGSZ || dca_en)) {
qp->pageshift = HNS_HW_PAGE_SHIFT;
return;
}
@@ -1334,7 +1334,7 @@ static void get_best_multi_region_pg_shift(struct hns_roce_device *hr_dev,
static int calc_qp_buff_size(struct hns_roce_device *hr_dev,
struct hns_roce_context *ctx,
- struct hns_roce_qp *qp)
+ struct hns_roce_qp *qp, bool dca_en)
{
struct hns_roce_wq *sq = &qp->sq;
struct hns_roce_wq *rq = &qp->rq;
@@ -1342,7 +1342,7 @@ static int calc_qp_buff_size(struct hns_roce_device *hr_dev,
unsigned int size;
qp->buf_size = 0;
- get_best_multi_region_pg_shift(hr_dev, ctx, qp);
+ get_best_multi_region_pg_shift(hr_dev, ctx, qp, dca_en);
page_size = 1 << qp->pageshift;
/* SQ WQE */
@@ -1384,7 +1384,7 @@ static inline bool check_qp_support_dca(struct hns_roce_dca_ctx *dca_ctx,
if (hns_attr &&
(hns_attr->comp_mask & HNSDV_QP_INIT_ATTR_MASK_QP_CREATE_FLAGS) &&
(hns_attr->create_flags & HNSDV_QP_CREATE_ENABLE_DCA_MODE))
- return true;
+ return dca_ctx->max_size > 0;
return false;
}
@@ -1408,9 +1408,12 @@ static int qp_alloc_wqe(struct ibv_qp_init_attr_ex *attr,
struct hns_roce_qp *qp, struct hns_roce_context *ctx)
{
struct hns_roce_device *hr_dev = to_hr_dev(ctx->ibv_ctx.context.device);
+ bool dca_en = check_qp_support_dca(&ctx->dca_ctx, attr, hns_attr);
+ int ret;
- if (calc_qp_buff_size(hr_dev, ctx, qp))
- return -EINVAL;
+ ret = calc_qp_buff_size(hr_dev, ctx, qp, dca_en);
+ if (ret)
+ return ret;
qp->sq.wrid = malloc(qp->sq.wqe_cnt * sizeof(uint64_t));
if (!qp->sq.wrid)
@@ -1428,19 +1431,18 @@ static int qp_alloc_wqe(struct ibv_qp_init_attr_ex *attr,
goto err_alloc;
}
- if (check_qp_support_dca(&ctx->dca_ctx, attr, hns_attr) &&
- ctx->dca_ctx.max_size > 0) {
+ if (check_qp_support_dca(&ctx->dca_ctx, attr, hns_attr)) {
/* when DCA is enabled, use a buffer list to store page addr */
qp->buf.buf = NULL;
qp->dca_wqe.max_cnt = hr_hw_page_count(qp->buf_size);
- qp->dca_wqe.shift = HNS_HW_PAGE_SHIFT;
+ qp->dca_wqe.shift = qp->pageshift;
qp->dca_wqe.bufs = calloc(qp->dca_wqe.max_cnt, sizeof(void *));
if (!qp->dca_wqe.bufs)
goto err_alloc;
verbs_debug(&ctx->ibv_ctx, "alloc DCA buf.\n");
} else {
if (hns_roce_alloc_buf(&qp->buf, qp->buf_size,
- HNS_HW_PAGE_SIZE))
+ 1 << qp->pageshift))
goto err_alloc;
}
--
2.33.0

55
0047-libhns-fix-missing-new-IO-support-for-DCA.patch Normal file
View File

@ -0,0 +1,55 @@
From 199b2f78ff9eeeb25acc78f9da495ae58877807a Mon Sep 17 00:00:00 2001
From: Chengchang Tang <tangchengchang@huawei.com>
Date: Tue, 30 Jan 2024 21:28:44 +0800
Subject: [PATCH 46/46] libhns: fix missing new IO support for DCA
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IBSL67
------------------------------------------------------------------
New IO related support has been missed for DCA.
Fixes: 9ab7600d832b ("libhns: Add support for attaching QP's WQE buffer")
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u_hw_v2.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index aadea7a..3137111 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -2191,6 +2191,8 @@ static int wc_start_poll_cq(struct ibv_cq_ex *current,
}
err = hns_roce_poll_one(ctx, &qp, cq, NULL);
+ if (qp && check_dca_detach_enable(qp))
+ dca_detach_qp_buf(ctx, qp);
start_poll_done:
if (err != V2_CQ_OK)
@@ -2210,6 +2212,8 @@ static int wc_next_poll_cq(struct ibv_cq_ex *current)
return hns_roce_poll_one_swc(cq, NULL);
err = hns_roce_poll_one(ctx, &qp, cq, NULL);
+ if (qp && check_dca_detach_enable(qp))
+ dca_detach_qp_buf(ctx, qp);
if (err != V2_CQ_OK)
return err;
@@ -2408,6 +2412,9 @@ init_rc_wqe(struct hns_roce_qp *qp, uint64_t wr_id, unsigned int opcode)
hr_reg_write_bool(wqe, RCWQE_SE, send_flags & IBV_SEND_SOLICITED);
hr_reg_clear(wqe, RCWQE_INLINE);
+ if (check_qp_dca_enable(qp))
+ fill_rc_dca_fields(qp->verbs_qp.qp.qp_num, wqe);
+
qp->sq.wrid[wqe_idx] = wr_id;
qp->cur_wqe = wqe;
--
2.33.0

245
0048-libzrdma-Fix-wqe-polarity-set-error.patch Normal file
View File

@ -0,0 +1,245 @@
From 232ae986da9c9995be0ae39bdf5b2145250c22c9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E6=9D=8E=E5=AF=8C=E8=89=B3?= <li.fuyan@zte.com.cn>
Date: Fri, 28 Mar 2025 11:04:33 +0800
Subject: [PATCH] Fix: wqe polarity set error
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: 李富艳 <li.fuyan@zte.com.cn>
---
providers/zrdma/zxdh_defs.h | 3 --
providers/zrdma/zxdh_hw.c | 62 +++++++-----------------------------
providers/zrdma/zxdh_verbs.h | 1 -
3 files changed, 11 insertions(+), 55 deletions(-)
diff --git a/providers/zrdma/zxdh_defs.h b/providers/zrdma/zxdh_defs.h
index eaf73ca..3863fb9 100644
--- a/providers/zrdma/zxdh_defs.h
+++ b/providers/zrdma/zxdh_defs.h
@@ -313,9 +313,6 @@
#define ZXDH_RING_FREE_QUANTA(_ring) \
(((_ring).size - ZXDH_RING_USED_QUANTA(_ring) - 1))
-#define ZXDH_SQ_RING_FREE_QUANTA(_ring) \
- (((_ring).size - ZXDH_RING_USED_QUANTA(_ring) - 257))
-
#define ZXDH_ATOMIC_RING_MOVE_HEAD(_ring, index, _retcode) \
{ \
index = ZXDH_RING_CURRENT_HEAD(_ring); \
diff --git a/providers/zrdma/zxdh_hw.c b/providers/zrdma/zxdh_hw.c
index ed577a9..073b198 100644
--- a/providers/zrdma/zxdh_hw.c
+++ b/providers/zrdma/zxdh_hw.c
@@ -20,13 +20,14 @@
#include <endian.h>
#define ERROR_CODE_VALUE 65
-static void qp_tx_psn_add(__u32 *x, __u32 y, __u16 mtu)
+static inline void qp_tx_psn_add(__u32 *x, __u32 y, __u16 mtu)
{
if (y == 0) {
*x = (*x + 1) & 0xffffff;
return;
}
- *x = (*x + ((y % mtu) ? (y / mtu + 1) : y / mtu)) & 0xffffff;
+ __u32 chunks = (y + mtu - 1) / mtu;
+ *x = (*x + chunks) & 0xffffff;
}
int zxdh_get_write_imm_split_switch(void)
@@ -95,26 +96,6 @@ static enum zxdh_status_code zxdh_nop_1(struct zxdh_qp *qp)
return 0;
}
-/**
- * zxdh_clr_wqes - clear next 128 sq entries
- * @qp: hw qp ptr
- * @qp_wqe_idx: wqe_idx
- */
-void zxdh_clr_wqes(struct zxdh_qp *qp, __u32 qp_wqe_idx)
-{
- __le64 *wqe;
- __u32 wqe_idx;
-
- if (!(qp_wqe_idx & 0x7F)) {
- wqe_idx = (qp_wqe_idx + 128) % qp->sq_ring.size;
- wqe = qp->sq_base[wqe_idx].elem;
- if (wqe_idx)
- memset(wqe, qp->swqe_polarity ? 0 : 0xFF, 0x1000);
- else
- memset(wqe, qp->swqe_polarity ? 0xFF : 0, 0x1000);
- }
-}
-
/**
* zxdh_qp_post_wr - ring doorbell
* @qp: hw qp ptr
@@ -197,14 +178,13 @@ __le64 *zxdh_qp_get_next_send_wqe(struct zxdh_qp *qp, __u32 *wqe_idx,
avail_quanta = ZXDH_MAX_SQ_WQES_PER_PAGE -
(ZXDH_RING_CURRENT_HEAD(qp->sq_ring) %
ZXDH_MAX_SQ_WQES_PER_PAGE);
- if (quanta <= avail_quanta) {
+ if (likely(quanta <= avail_quanta)) {
/* WR fits in current chunk */
- if (quanta > ZXDH_SQ_RING_FREE_QUANTA(qp->sq_ring))
+ if (unlikely(quanta > ZXDH_RING_FREE_QUANTA(qp->sq_ring)))
return NULL;
} else {
/* Need to pad with NOP */
- if (quanta + avail_quanta >
- ZXDH_SQ_RING_FREE_QUANTA(qp->sq_ring))
+ if (quanta + avail_quanta > ZXDH_RING_FREE_QUANTA(qp->sq_ring))
return NULL;
for (i = 0; i < avail_quanta; i++) {
@@ -287,8 +267,6 @@ zxdh_post_rdma_write(struct zxdh_qp *qp, struct zxdh_post_sq_info *info,
if (!wqe)
return ZXDH_ERR_QP_TOOMANY_WRS_POSTED;
- zxdh_clr_wqes(qp, wqe_idx);
-
if (op_info->num_lo_sges) {
set_64bit_val(
wqe, 16,
@@ -635,8 +613,6 @@ static enum zxdh_status_code zxdh_post_rdma_read(struct zxdh_qp *qp,
if (!wqe)
return ZXDH_ERR_QP_TOOMANY_WRS_POSTED;
- zxdh_clr_wqes(qp, wqe_idx);
-
addl_frag_cnt =
op_info->num_lo_sges > 1 ? (op_info->num_lo_sges - 1) : 0;
local_fence |= info->local_fence;
@@ -817,8 +793,6 @@ enum zxdh_status_code zxdh_rc_send(struct zxdh_qp *qp,
if (!wqe)
return ZXDH_ERR_QP_TOOMANY_WRS_POSTED;
- zxdh_clr_wqes(qp, wqe_idx);
-
read_fence |= info->read_fence;
addl_frag_cnt = op_info->num_sges > 1 ? (op_info->num_sges - 1) : 0;
if (op_info->num_sges) {
@@ -975,7 +949,7 @@ enum zxdh_status_code zxdh_ud_send(struct zxdh_qp *qp,
if (ret_code)
return ret_code;
- if (quanta > ZXDH_SQ_RING_FREE_QUANTA(qp->sq_ring))
+ if (quanta > ZXDH_RING_FREE_QUANTA(qp->sq_ring))
return ZXDH_ERR_QP_TOOMANY_WRS_POSTED;
wqe_idx = ZXDH_RING_CURRENT_HEAD(qp->sq_ring);
@@ -989,8 +963,6 @@ enum zxdh_status_code zxdh_ud_send(struct zxdh_qp *qp,
qp->sq_wrtrk_array[wqe_idx].wr_len = total_size;
qp->sq_wrtrk_array[wqe_idx].quanta = quanta;
- zxdh_clr_wqes(qp, wqe_idx);
-
read_fence |= info->read_fence;
addl_frag_cnt = op_info->num_sges > 1 ? (op_info->num_sges - 1) : 0;
hdr = FIELD_PREP(ZXDHQPSQ_VALID, qp->swqe_polarity) |
@@ -1281,8 +1253,6 @@ enum zxdh_status_code zxdh_inline_rdma_write(struct zxdh_qp *qp,
if (!wqe)
return ZXDH_ERR_QP_TOOMANY_WRS_POSTED;
- zxdh_clr_wqes(qp, wqe_idx);
-
read_fence |= info->read_fence;
hdr = FIELD_PREP(ZXDHQPSQ_VALID, qp->swqe_polarity) |
FIELD_PREP(ZXDHQPSQ_OPCODE, info->op_type) |
@@ -1293,7 +1263,7 @@ enum zxdh_status_code zxdh_inline_rdma_write(struct zxdh_qp *qp,
FIELD_PREP(ZXDHQPSQ_IMMDATAFLAG, imm_data_flag) |
FIELD_PREP(ZXDHQPSQ_WRITE_INLINEDATAFLAG, 1) |
FIELD_PREP(ZXDHQPSQ_WRITE_INLINEDATALEN, op_info->len) |
- FIELD_PREP(ZXDHQPSQ_ADDFRAGCNT, quanta - 1) |
+ FIELD_PREP(ZXDHQPSQ_ADDFRAGCNT, (__u16)(quanta - 1)) |
FIELD_PREP(ZXDHQPSQ_REMSTAG, op_info->rem_addr.stag);
set_64bit_val(wqe, 24,
FIELD_PREP(ZXDHQPSQ_FRAG_TO, op_info->rem_addr.tag_off));
@@ -1355,8 +1325,6 @@ enum zxdh_status_code zxdh_rc_inline_send(struct zxdh_qp *qp,
if (!wqe)
return ZXDH_ERR_QP_TOOMANY_WRS_POSTED;
- zxdh_clr_wqes(qp, wqe_idx);
-
read_fence |= info->read_fence;
hdr = FIELD_PREP(ZXDHQPSQ_VALID, qp->swqe_polarity) |
FIELD_PREP(ZXDHQPSQ_OPCODE, info->op_type) |
@@ -1364,7 +1332,7 @@ enum zxdh_status_code zxdh_rc_inline_send(struct zxdh_qp *qp,
FIELD_PREP(ZXDHQPSQ_LOCALFENCE, info->local_fence) |
FIELD_PREP(ZXDHQPSQ_READFENCE, read_fence) |
FIELD_PREP(ZXDHQPSQ_SOLICITED, info->solicited) |
- FIELD_PREP(ZXDHQPSQ_ADDFRAGCNT, quanta - 1) |
+ FIELD_PREP(ZXDHQPSQ_ADDFRAGCNT, (__u16)(quanta - 1)) |
FIELD_PREP(ZXDHQPSQ_IMMDATAFLAG, imm_data_flag) |
FIELD_PREP(ZXDHQPSQ_REMSTAG, info->stag_to_inv);
set_64bit_val(wqe, 24,
@@ -1430,7 +1398,7 @@ enum zxdh_status_code zxdh_ud_inline_send(struct zxdh_qp *qp,
quanta = qp->wqe_ops.iw_inline_data_size_to_quanta(op_info->len,
imm_data_flag);
- if (quanta > ZXDH_SQ_RING_FREE_QUANTA(qp->sq_ring))
+ if (quanta > ZXDH_RING_FREE_QUANTA(qp->sq_ring))
return ZXDH_ERR_QP_TOOMANY_WRS_POSTED;
wqe_idx = ZXDH_RING_CURRENT_HEAD(qp->sq_ring);
@@ -1444,8 +1412,6 @@ enum zxdh_status_code zxdh_ud_inline_send(struct zxdh_qp *qp,
qp->sq_wrtrk_array[wqe_idx].wr_len = op_info->len;
qp->sq_wrtrk_array[wqe_idx].quanta = quanta;
- zxdh_clr_wqes(qp, wqe_idx);
-
read_fence |= info->read_fence;
hdr = FIELD_PREP(ZXDHQPSQ_VALID, qp->swqe_polarity) |
FIELD_PREP(ZXDHQPSQ_OPCODE, info->op_type) |
@@ -1454,7 +1420,7 @@ enum zxdh_status_code zxdh_ud_inline_send(struct zxdh_qp *qp,
FIELD_PREP(ZXDHQPSQ_IMMDATAFLAG, imm_data_flag) |
FIELD_PREP(ZXDHQPSQ_UD_INLINEDATAFLAG, 1) |
FIELD_PREP(ZXDHQPSQ_UD_INLINEDATALEN, op_info->len) |
- FIELD_PREP(ZXDHQPSQ_UD_ADDFRAGCNT, quanta - 1) |
+ FIELD_PREP(ZXDHQPSQ_UD_ADDFRAGCNT, (__u16)(quanta - 1)) |
FIELD_PREP(ZXDHQPSQ_AHID, op_info->ah_id);
set_64bit_val(wqe_base, 24,
FIELD_PREP(ZXDHQPSQ_DESTQPN, op_info->dest_qp) |
@@ -1572,8 +1538,6 @@ enum zxdh_status_code zxdh_stag_local_invalidate(struct zxdh_qp *qp,
if (!wqe)
return ZXDH_ERR_QP_TOOMANY_WRS_POSTED;
- zxdh_clr_wqes(qp, wqe_idx);
-
set_64bit_val(wqe, 16, 0);
hdr = FIELD_PREP(ZXDHQPSQ_VALID, qp->swqe_polarity) |
@@ -1618,8 +1582,6 @@ enum zxdh_status_code zxdh_mw_bind(struct zxdh_qp *qp,
if (!wqe)
return ZXDH_ERR_QP_TOOMANY_WRS_POSTED;
- zxdh_clr_wqes(qp, wqe_idx);
-
if (op_info->ena_writes) {
access = (op_info->ena_reads << 2) |
(op_info->ena_writes << 3) | (1 << 1) | access;
@@ -2391,8 +2353,6 @@ enum zxdh_status_code zxdh_nop(struct zxdh_qp *qp, __u64 wr_id, bool signaled,
if (!wqe)
return ZXDH_ERR_QP_TOOMANY_WRS_POSTED;
- zxdh_clr_wqes(qp, wqe_idx);
-
set_64bit_val(wqe, 0, 0);
set_64bit_val(wqe, 8, 0);
set_64bit_val(wqe, 16, 0);
diff --git a/providers/zrdma/zxdh_verbs.h b/providers/zrdma/zxdh_verbs.h
index 69a98cc..40aa7bb 100644
--- a/providers/zrdma/zxdh_verbs.h
+++ b/providers/zrdma/zxdh_verbs.h
@@ -596,7 +596,6 @@ int zxdh_qp_round_up(__u32 wqdepth);
int zxdh_cq_round_up(__u32 wqdepth);
void zxdh_qp_push_wqe(struct zxdh_qp *qp, __le64 *wqe, __u16 quanta,
__u32 wqe_idx, bool post_sq);
-void zxdh_clr_wqes(struct zxdh_qp *qp, __u32 qp_wqe_idx);
void zxdh_get_srq_wqe_shift(struct zxdh_dev_attrs *dev_attrs, __u32 sge,
__u8 *shift);
--
2.27.0

304
0049-libzrdma-Add-interface-aligned-with-kernel.patch Normal file
View File

@ -0,0 +1,304 @@
From da8370c2360deb73af7a211bec2be76b025cb5d3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E6=9D=8E=E5=AF=8C=E8=89=B3?= <li.fuyan@zte.com.cn>
Date: Fri, 28 Mar 2025 11:36:50 +0800
Subject: [PATCH] libzrdma:Add interface aligned with kernel
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: 李富艳 <li.fuyan@zte.com.cn>
---
kernel-headers/rdma/zxdh-abi.h | 23 ++++++++-------
providers/zrdma/main.c | 34 +++++++++++++----------
providers/zrdma/zxdh_defs.h | 8 +++---
providers/zrdma/zxdh_devids.h | 9 ++++++
providers/zrdma/zxdh_hw.c | 9 ++++--
providers/zrdma/zxdh_verbs.h | 51 ++++++----------------------------
6 files changed, 60 insertions(+), 74 deletions(-)
diff --git a/kernel-headers/rdma/zxdh-abi.h b/kernel-headers/rdma/zxdh-abi.h
index 665f874..59c0160 100644
--- a/kernel-headers/rdma/zxdh-abi.h
+++ b/kernel-headers/rdma/zxdh-abi.h
@@ -6,10 +6,9 @@
#include <linux/types.h>
-/* zxdh must support legacy GEN_1 i40iw kernel
- * and user-space whose last ABI ver is 5
- */
+/* user-space whose last ABI ver is 5 */
#define ZXDH_ABI_VER 5
+#define ZXDH_CONTEXT_VER_V1 5
enum zxdh_memreg_type {
ZXDH_MEMREG_TYPE_MEM = 0,
@@ -35,7 +34,7 @@ struct zxdh_alloc_ucontext_resp {
__u32 wq_size; /* size of the WQs (SQ+RQ) in the mmaped area */
__u8 kernel_ver;
__u8 db_addr_type;
- __u8 rsvd[2];
+ __u16 rdma_tool_flags;
__aligned_u64 feature_flags;
__aligned_u64 sq_db_mmap_key;
__aligned_u64 cq_db_mmap_key;
@@ -51,8 +50,8 @@ struct zxdh_alloc_ucontext_resp {
__u32 min_hw_cq_size;
__u32 max_hw_cq_size;
__u16 max_hw_sq_chunk;
- __u8 hw_rev;
- __u8 rsvd2;
+ __u8 rsvd;
+ __u8 chip_rev;
};
struct zxdh_alloc_pd_resp {
@@ -82,13 +81,13 @@ struct zxdh_create_srq_req {
};
struct zxdh_mem_reg_req {
- __u16 reg_type; /* enum zxdh_memreg_type */
- __u16 cq_pages;
- __u16 rq_pages;
- __u16 sq_pages;
- __u16 srq_pages;
+ __u32 reg_type; /* enum zxdh_memreg_type */
+ __u32 cq_pages;
+ __u32 rq_pages;
+ __u32 sq_pages;
+ __u32 srq_pages;
__u16 srq_list_pages;
- __u8 rsvd[4];
+ __u8 rsvd[2];
};
struct zxdh_reg_mr_resp {
diff --git a/providers/zrdma/main.c b/providers/zrdma/main.c
index e25a1a2..4626a21 100644
--- a/providers/zrdma/main.c
+++ b/providers/zrdma/main.c
@@ -22,6 +22,12 @@ static const struct verbs_match_ent hca_table[] = {
ZXDH_HCA(PCI_VENDOR_ID_ZXDH_EVB, ZXDH_DEV_ID_ADAPTIVE_EVB_VF),
ZXDH_HCA(PCI_VENDOR_ID_ZXDH_E312, ZXDH_DEV_ID_ADAPTIVE_E312_PF),
ZXDH_HCA(PCI_VENDOR_ID_ZXDH_E312, ZXDH_DEV_ID_ADAPTIVE_E312_VF),
+ ZXDH_HCA(PCI_VENDOR_ID_ZXDH_E310, ZXDH_DEV_ID_ADAPTIVE_E310_PF),
+ ZXDH_HCA(PCI_VENDOR_ID_ZXDH_E310, ZXDH_DEV_ID_ADAPTIVE_E310_VF),
+ ZXDH_HCA(PCI_VENDOR_ID_ZXDH_E310_RDMA, ZXDH_DEV_ID_ADAPTIVE_E310_RDMA_PF),
+ ZXDH_HCA(PCI_VENDOR_ID_ZXDH_E310_RDMA, ZXDH_DEV_ID_ADAPTIVE_E310_RDMA_VF),
+ ZXDH_HCA(PCI_VENDOR_ID_ZXDH_E316, ZXDH_DEV_ID_ADAPTIVE_E316_PF),
+ ZXDH_HCA(PCI_VENDOR_ID_ZXDH_E316, ZXDH_DEV_ID_ADAPTIVE_E316_VF),
ZXDH_HCA(PCI_VENDOR_ID_ZXDH_X512, ZXDH_DEV_ID_ADAPTIVE_X512_PF),
ZXDH_HCA(PCI_VENDOR_ID_ZXDH_X512, ZXDH_DEV_ID_ADAPTIVE_X512_VF),
{}
@@ -100,7 +106,6 @@ static struct verbs_context *zxdh_ualloc_context(struct ibv_device *ibdev,
struct zxdh_get_context cmd;
struct zxdh_get_context_resp resp = {};
__u64 sq_db_mmap_key, cq_db_mmap_key;
- __u8 user_ver = ZXDH_ABI_VER;
iwvctx = verbs_init_and_alloc_context(ibdev, cmd_fd, iwvctx, ibv_ctx,
RDMA_DRIVER_ZXDH);
@@ -109,22 +114,16 @@ static struct verbs_context *zxdh_ualloc_context(struct ibv_device *ibdev,
zxdh_set_debug_mask();
iwvctx->zxdh_write_imm_split_switch = zxdh_get_write_imm_split_switch();
- cmd.userspace_ver = user_ver;
+
+ cmd.userspace_ver = ZXDH_CONTEXT_VER_V1;
if (ibv_cmd_get_context(&iwvctx->ibv_ctx,
(struct ibv_get_context *)&cmd, sizeof(cmd),
- &resp.ibv_resp, sizeof(resp))) {
- cmd.userspace_ver = 4;
- if (ibv_cmd_get_context(
- &iwvctx->ibv_ctx, (struct ibv_get_context *)&cmd,
- sizeof(cmd), &resp.ibv_resp, sizeof(resp)))
- goto err_free;
- user_ver = cmd.userspace_ver;
- }
+ &resp.ibv_resp, sizeof(resp)))
+ goto err_free;
verbs_set_ops(&iwvctx->ibv_ctx, &zxdh_uctx_ops);
iwvctx->dev_attrs.feature_flags = resp.feature_flags;
- iwvctx->dev_attrs.hw_rev = resp.hw_rev;
iwvctx->dev_attrs.max_hw_wq_frags = resp.max_hw_wq_frags;
iwvctx->dev_attrs.max_hw_read_sges = resp.max_hw_read_sges;
iwvctx->dev_attrs.max_hw_inline = resp.max_hw_inline;
@@ -135,11 +134,20 @@ static struct verbs_context *zxdh_ualloc_context(struct ibv_device *ibdev,
iwvctx->dev_attrs.max_hw_sq_chunk = resp.max_hw_sq_chunk;
iwvctx->dev_attrs.max_hw_cq_size = resp.max_hw_cq_size;
iwvctx->dev_attrs.min_hw_cq_size = resp.min_hw_cq_size;
- iwvctx->abi_ver = user_ver;
+ iwvctx->abi_ver = ZXDH_ABI_VER;
+ iwvctx->dev_attrs.chip_rev = resp.chip_rev;
+ iwvctx->dev_attrs.rdma_tool_flags = resp.rdma_tool_flags;
sq_db_mmap_key = resp.sq_db_mmap_key;
cq_db_mmap_key = resp.cq_db_mmap_key;
+ iwvctx->dev_attrs.db_addr_type = resp.db_addr_type;
+ iwvctx->dev_attrs.sq_db_pa = resp.sq_db_pa;
+ iwvctx->dev_attrs.cq_db_pa = resp.cq_db_pa;
+
+ if (iwvctx->dev_attrs.db_addr_type != ZXDH_DB_ADDR_BAR)
+ goto err_free;
+
iwvctx->sq_db = zxdh_mmap(cmd_fd, sq_db_mmap_key);
if (iwvctx->sq_db == MAP_FAILED)
goto err_free;
@@ -160,10 +168,8 @@ static struct verbs_context *zxdh_ualloc_context(struct ibv_device *ibdev,
iwvctx->iwupd = container_of(ibv_pd, struct zxdh_upd, ibv_pd);
add_private_ops(iwvctx);
return &iwvctx->ibv_ctx;
-
err_free:
free(iwvctx);
-
return NULL;
}
diff --git a/providers/zrdma/zxdh_defs.h b/providers/zrdma/zxdh_defs.h
index 3863fb9..8772e7b 100644
--- a/providers/zrdma/zxdh_defs.h
+++ b/providers/zrdma/zxdh_defs.h
@@ -389,8 +389,8 @@ static inline void db_wr32(__u32 val, __u32 *wqe_word)
*wqe_word = val;
}
-#define read_wqe_need_split(pre_cal_psn, next_psn) \
- (((pre_cal_psn < next_psn) && (pre_cal_psn != 0)) || \
- ((next_psn <= 0x7FFFFF) && (pre_cal_psn > 0x800000)))
-
+#define read_wqe_need_split(pre_cal_psn, next_psn, chip_rev) \
+ (!(chip_rev == 2) && \
+ (((pre_cal_psn < next_psn) && (pre_cal_psn != 0)) || \
+ ((next_psn <= 0x7FFFFF) && (pre_cal_psn > 0x800000))))
#endif /* ZXDH_DEFS_H */
diff --git a/providers/zrdma/zxdh_devids.h b/providers/zrdma/zxdh_devids.h
index ac23124..3430f5f 100644
--- a/providers/zrdma/zxdh_devids.h
+++ b/providers/zrdma/zxdh_devids.h
@@ -6,12 +6,21 @@
/* ZXDH VENDOR ID */
#define PCI_VENDOR_ID_ZXDH_EVB 0x16c3
#define PCI_VENDOR_ID_ZXDH_E312 0x1cf2
+#define PCI_VENDOR_ID_ZXDH_E310 0x1cf2
+#define PCI_VENDOR_ID_ZXDH_E310_RDMA 0x1cf2
+#define PCI_VENDOR_ID_ZXDH_E316 0x1cf2
#define PCI_VENDOR_ID_ZXDH_X512 0x1cf2
/* ZXDH Devices ID */
#define ZXDH_DEV_ID_ADAPTIVE_EVB_PF 0x8040 /* ZXDH EVB PF DEVICE ID*/
#define ZXDH_DEV_ID_ADAPTIVE_EVB_VF 0x8041 /* ZXDH EVB VF DEVICE ID*/
#define ZXDH_DEV_ID_ADAPTIVE_E312_PF 0x8049 /* ZXDH E312 PF DEVICE ID*/
#define ZXDH_DEV_ID_ADAPTIVE_E312_VF 0x8060 /* ZXDH E312 VF DEVICE ID*/
+#define ZXDH_DEV_ID_ADAPTIVE_E310_PF 0x8061 /* ZXDH E310 PF DEVICE ID*/
+#define ZXDH_DEV_ID_ADAPTIVE_E310_VF 0x8062 /* ZXDH E310 VF DEVICE ID*/
+#define ZXDH_DEV_ID_ADAPTIVE_E310_RDMA_PF 0x8084 /* ZXDH E310_RDMA PF DEVICE ID*/
+#define ZXDH_DEV_ID_ADAPTIVE_E310_RDMA_VF 0x8085 /* ZXDH E310_RDMA VF DEVICE ID*/
+#define ZXDH_DEV_ID_ADAPTIVE_E316_PF 0x807e /* ZXDH E316 PF DEVICE ID*/
+#define ZXDH_DEV_ID_ADAPTIVE_E316_VF 0x807f /* ZXDH E316 VF DEVICE ID*/
#define ZXDH_DEV_ID_ADAPTIVE_X512_PF 0x806B /* ZXDH X512 PF DEVICE ID*/
#define ZXDH_DEV_ID_ADAPTIVE_X512_VF 0x806C /* ZXDH X512 VF DEVICE ID*/
#endif /* ZXDH_DEVIDS_H */
diff --git a/providers/zrdma/zxdh_hw.c b/providers/zrdma/zxdh_hw.c
index 073b198..99489dc 100644
--- a/providers/zrdma/zxdh_hw.c
+++ b/providers/zrdma/zxdh_hw.c
@@ -703,8 +703,12 @@ enum zxdh_status_code zxdh_rdma_read(struct zxdh_qp *qp,
struct zxdh_post_sq_info split_part2_info = { 0 };
struct zxdh_rdma_read *op_info;
enum zxdh_status_code ret_code;
+ struct zxdh_uqp *iwuqp;
+ struct zxdh_uvcontext *iwvctx;
__u32 i, total_size = 0, pre_cal_psn = 0;
-
+ iwuqp = container_of(qp, struct zxdh_uqp, qp);
+ iwvctx = container_of(iwuqp->vqp.qp.context, struct zxdh_uvcontext,
+ ibv_ctx.context);
op_info = &info->op.rdma_read;
if (qp->max_sq_frag_cnt < op_info->num_lo_sges)
return ZXDH_ERR_INVALID_FRAG_COUNT;
@@ -720,7 +724,8 @@ enum zxdh_status_code zxdh_rdma_read(struct zxdh_qp *qp,
op_info->rem_addr.len = total_size;
pre_cal_psn = qp->next_psn;
qp_tx_psn_add(&pre_cal_psn, total_size, qp->mtu);
- if (read_wqe_need_split(pre_cal_psn, qp->next_psn)) {
+ if (read_wqe_need_split(pre_cal_psn, qp->next_psn,
+ iwvctx->dev_attrs.chip_rev)) {
split_two_part_info(qp, info, qp->next_psn, pre_cal_psn,
&split_part1_info, &split_part2_info);
ret_code = zxdh_post_rdma_read(qp, &split_part1_info, post_sq,
diff --git a/providers/zrdma/zxdh_verbs.h b/providers/zrdma/zxdh_verbs.h
index 40aa7bb..1a26cf4 100644
--- a/providers/zrdma/zxdh_verbs.h
+++ b/providers/zrdma/zxdh_verbs.h
@@ -71,47 +71,13 @@
#define ZXDH_WQEALLOC_WQE_DESC_INDEX GENMASK(31, 20)
-enum zxdh_device_caps_const {
- ZXDH_WQE_SIZE = 4,
- ZXDH_SRQE_SIZE = 2,
- ZXDH_CQP_WQE_SIZE = 8,
- ZXDH_CQE_SIZE = 8,
- ZXDH_EXTENDED_CQE_SIZE = 8,
- ZXDH_AEQE_SIZE = 2,
- ZXDH_CEQE_SIZE = 1,
- ZXDH_CQP_CTX_SIZE = 8,
- ZXDH_SHADOW_AREA_SIZE = 8,
- ZXDH_GATHER_STATS_BUF_SIZE = 1024,
- ZXDH_MIN_IW_QP_ID = 0,
- ZXDH_QUERY_FPM_BUF_SIZE = 176,
- ZXDH_COMMIT_FPM_BUF_SIZE = 176,
- ZXDH_MAX_IW_QP_ID = 262143,
- ZXDH_MIN_CEQID = 0,
- ZXDH_MAX_CEQID = 1023,
- ZXDH_CEQ_MAX_COUNT = ZXDH_MAX_CEQID + 1,
- ZXDH_MIN_CQID = 0,
- ZXDH_MAX_CQID = 524287,
- ZXDH_MIN_AEQ_ENTRIES = 1,
- ZXDH_MAX_AEQ_ENTRIES = 524287,
- ZXDH_MIN_CEQ_ENTRIES = 1,
- ZXDH_MAX_CEQ_ENTRIES = 262143,
- ZXDH_MIN_CQ_SIZE = 1,
- ZXDH_MAX_CQ_SIZE = 1048575,
- ZXDH_DB_ID_ZERO = 0,
- ZXDH_MAX_WQ_FRAGMENT_COUNT = 13,
- ZXDH_MAX_SGE_RD = 13,
- ZXDH_MAX_OUTBOUND_MSG_SIZE = 2147483647,
- ZXDH_MAX_INBOUND_MSG_SIZE = 2147483647,
- ZXDH_MAX_PUSH_PAGE_COUNT = 1024,
- ZXDH_MAX_PE_ENA_VF_COUNT = 32,
- ZXDH_MAX_VF_FPM_ID = 47,
- ZXDH_MAX_SQ_PAYLOAD_SIZE = 2147483648,
- ZXDH_MAX_INLINE_DATA_SIZE = 217,
- ZXDH_MAX_WQ_ENTRIES = 32768,
- ZXDH_Q2_BUF_SIZE = 256,
- ZXDH_QP_CTX_SIZE = 256,
- ZXDH_MAX_PDS = 262144,
-};
+#define ZXDH_SRQE_SIZE 2
+#define ZXDH_CQE_SIZE 8
+#define ZXDH_EXTENDED_CQE_SIZE 8
+#define ZXDH_MAX_INLINE_DATA_SIZE 217
+#define ZXDH_MAX_SQ_PAYLOAD_SIZE 2147483648
+#define ZXDH_MIN_CQ_SIZE 1
+#define ZXDH_MAX_CQ_SIZE 2097152
enum zxdh_addressing_type {
ZXDH_ADDR_TYPE_ZERO_BASED = 0,
@@ -394,8 +360,9 @@ struct zxdh_dev_attrs {
__u32 max_hw_cq_size;
__u16 max_hw_sq_chunk;
__u32 max_hw_srq_wr;
- __u8 hw_rev;
__u8 db_addr_type;
+ __u8 chip_rev;
+ __u16 rdma_tool_flags;
};
struct zxdh_hw_attrs {
--
2.27.0

241
0050-libzrdma-Add-poll-cqe-error-to-Failed-status.patch Normal file
View File

@ -0,0 +1,241 @@
From 3a0b295e98d9557c65274424dc23b74de4aef8d2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E6=9D=8E=E5=AF=8C=E8=89=B3?= <li.fuyan@zte.com.cn>
Date: Fri, 28 Mar 2025 15:13:32 +0800
Subject: [PATCH] libzrdma:Add poll cqe error to Failed status
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: 李富艳 <li.fuyan@zte.com.cn>
---
providers/zrdma/zxdh_verbs.c | 68 +++++++++++++--------
providers/zrdma/zxdh_verbs.h | 113 +++++++++++++++++++++++++++++++++++
2 files changed, 156 insertions(+), 25 deletions(-)
diff --git a/providers/zrdma/zxdh_verbs.c b/providers/zrdma/zxdh_verbs.c
index 93cf705..f67f8c7 100644
--- a/providers/zrdma/zxdh_verbs.c
+++ b/providers/zrdma/zxdh_verbs.c
@@ -616,30 +616,50 @@ int zxdh_umodify_cq(struct ibv_cq *cq, struct ibv_modify_cq_attr *attr)
}
static enum ibv_wc_status
-zxdh_flush_err_to_ib_wc_status(enum zxdh_flush_opcode opcode)
+zxdh_err_to_ib_wc_status(__u32 opcode)
{
switch (opcode) {
- case FLUSH_PROT_ERR:
+ case ZXDH_RX_WQE_LEN_ERR:
+ return IBV_WC_LOC_LEN_ERR;
+ case ZXDH_TX_ACK_SYS_TOP_VADDR_LEN_CHECK_ERR:
+ case ZXDH_TX_ACK_SYS_TOP_LKEY_CHECK_ERR:
+ case ZXDH_TX_ACK_SYS_TOP_ACCESS_RIGHT_CHECK_ERR:
+ case ZXDH_RX_MR_MW_STATE_FREE_ERR:
+ case ZXDH_RX_MR_MW_STATE_INVALID_ERR:
+ case ZXDH_RX_MR_MW_PD_CHECK_ERR:
+ case ZXDH_RX_MR_MW_KEY_CHECK_ERR:
+ case ZXDH_RX_MR_MW_STAG_INDEX_CHECK_ERR:
+ case ZXDH_RX_MR_MW_BOUNDARY_CHECK_ERR:
+ case ZXDH_RX_MR_MW_0STAG_INDEX_CHECK_ERR:
+ case ZXDH_RX_MW_STATE_INVALID_ERR:
+ case ZXDH_RX_MW_PD_CHECK_ERR:
+ case ZXDH_RX_MW_STAG_INDEX_CHECK_ERR:
+ case ZXDH_RX_MW_SHARE_MR_CHECK_ERR:
+ case ZXDH_RX_MR_PD_CHECK_ERR:
+ case ZXDH_RX_MR_SHARE_MR_CHECK_ERR:
+ case ZXDH_RX_MR_MW_ACCESS_CHECK_ERR:
return IBV_WC_LOC_PROT_ERR;
- case FLUSH_REM_ACCESS_ERR:
+ case ZXDH_TX_PARSE_TOP_WQE_FLUSH:
+ return IBV_WC_WR_FLUSH_ERR;
+ case ZXDH_TX_ACK_SYS_TOP_NAK_INVALID_REQ:
+ return IBV_WC_REM_INV_REQ_ERR;
+ case ZXDH_TX_ACK_SYS_TOP_NAK_REMOTE_ACCESS_ERR:
+ case ZXDH_RX_MW_RKEY_CHECK_ERR:
+ case ZXDH_RX_MR_RKEY_CHECK_ERR:
return IBV_WC_REM_ACCESS_ERR;
- case FLUSH_LOC_QP_OP_ERR:
- return IBV_WC_LOC_QP_OP_ERR;
- case FLUSH_REM_OP_ERR:
+ case ZXDH_TX_ACK_SYS_TOP_NAK_REMOTE_OPERATIONAL_ERR:
return IBV_WC_REM_OP_ERR;
- case FLUSH_LOC_LEN_ERR:
- return IBV_WC_LOC_LEN_ERR;
- case FLUSH_GENERAL_ERR:
- return IBV_WC_WR_FLUSH_ERR;
- case FLUSH_RETRY_EXC_ERR:
+ case ZXDH_TX_ACK_SYS_TOP_NAK_RETRY_LIMIT:
+ case ZXDH_TX_ACK_SYS_TOP_READ_RETRY_LIMIT:
+ case ZXDH_TX_ACK_SYS_TOP_TIMEOUT_RETRY_LIMIT:
return IBV_WC_RETRY_EXC_ERR;
- case FLUSH_MW_BIND_ERR:
- return IBV_WC_MW_BIND_ERR;
- case FLUSH_REM_INV_REQ_ERR:
- return IBV_WC_REM_INV_REQ_ERR;
- case FLUSH_FATAL_ERR:
- default:
+ case ZXDH_TX_ACK_SYS_TOP_RNR_RETRY_LIMIT:
+ return IBV_WC_RNR_RETRY_EXC_ERR;
+ case ZXDH_TX_PARSE_TOP_AXI_ERR:
+ case ZXDH_RX_AXI_RESP_ERR:
return IBV_WC_FATAL_ERR;
+ default:
+ return IBV_WC_GENERAL_ERR;
}
}
@@ -656,10 +676,9 @@ static inline void zxdh_process_cqe_ext(struct zxdh_cq_poll_info *cur_cqe)
ibvcq_ex->wr_id = cur_cqe->wr_id;
if (cur_cqe->error)
ibvcq_ex->status =
- (cur_cqe->comp_status == ZXDH_COMPL_STATUS_FLUSHED) ?
- zxdh_flush_err_to_ib_wc_status(
- cur_cqe->minor_err) :
- IBV_WC_GENERAL_ERR;
+ zxdh_err_to_ib_wc_status(
+ cur_cqe->major_err << 16 |
+ cur_cqe->minor_err);
else
ibvcq_ex->status = IBV_WC_SUCCESS;
}
@@ -683,10 +702,9 @@ static inline void zxdh_process_cqe(struct ibv_wc *entry,
if (cur_cqe->error) {
entry->status =
- (cur_cqe->comp_status == ZXDH_COMPL_STATUS_FLUSHED) ?
- zxdh_flush_err_to_ib_wc_status(
- cur_cqe->minor_err) :
- IBV_WC_GENERAL_ERR;
+ zxdh_err_to_ib_wc_status(
+ cur_cqe->major_err << 16 |
+ cur_cqe->minor_err);
entry->vendor_err =
cur_cqe->major_err << 16 | cur_cqe->minor_err;
} else {
diff --git a/providers/zrdma/zxdh_verbs.h b/providers/zrdma/zxdh_verbs.h
index 1a26cf4..e3974c1 100644
--- a/providers/zrdma/zxdh_verbs.h
+++ b/providers/zrdma/zxdh_verbs.h
@@ -146,6 +146,119 @@ enum zxdh_page_size {
ZXDH_PAGE_SIZE_1G = 18,
};
+enum zxdh_rdmatx_parse_top_err {
+ ZXDH_TX_PARSE_TOP_AXI_ERR = 0x1,
+ ZXDH_TX_PARSE_TOP_WQE_FLUSH = 0x10001,
+ ZXDH_TX_PARSE_TOP_ORD_ERR = 0x20020,
+ ZXDH_TX_PARSE_TOP_OPCODE_ERR_FLAG = 0x20021,
+ ZXDH_TX_PARSE_TOP_CQP_STATE_AXI_ERR = 0x20022,
+ ZXDH_TX_PARSE_TOP_WQE_LEN_ERR = 0x20023,
+ ZXDH_TX_PARSE_TOP_DATA_LEN_ERR = 0x20024,
+ ZXDH_TX_PARSE_TOP_AH_VALID0_ERR = 0x20025,
+ ZXDH_TX_PARSE_TOP_UD_PDINDEX_ERR = 0x20026,
+ ZXDH_TX_PARSE_TOP_QP_STATE_ERR = 0x20027,
+ ZXDH_TX_PARSE_TOP_SERVICE_TYPE_ERR = 0x20028,
+ ZXDH_TX_PARSE_TOP_UD_PAYLOAD_ERR = 0x20029,
+ ZXDH_TX_PARSE_TOP_WQE_LEN0_ERR = 0x2002a,
+ ZXDH_TX_PARSE_TOP_WQE_DEFICIENT_CLR_ERR = 0x2002b,
+ ZXDH_TX_PARSE_TOP_IMMDT_ERR = 0x2002c,
+ ZXDH_TX_PARSE_TOP_FRAGMENT_LENGTH_ERR = 0x2009f,
+ ZXDH_TX_PARSE_TOP_MRTE_STATE_ERR = 0x90091,
+ ZXDH_TX_PARSE_TOP_QP_CHECK_ERR = 0x90092,
+ ZXDH_TX_PARSE_TOP_PD_CHECK_ERR = 0x90093,
+ ZXDH_TX_PARSE_TOP_LKEY_CHECK_ERR = 0x90094,
+ ZXDH_TX_PARSE_TOP_STAG_INDEX_CHECK_ERR = 0x90095,
+ ZXDH_TX_PARSE_TOP_VADDR_LEN_CHECK_ERR = 0x90096,
+ ZXDH_TX_PARSE_TOP_ACCESS_RIGHT_CHECK_ERR = 0x90097,
+ ZXDH_TX_PARSE_TOP_STAG_INDEX_CHECK_ZERO_ERR = 0x90098,
+};
+
+enum zxdh_rdmatx_ack_sys_top_err {
+ ZXDH_TX_ACK_SYS_TOP_NVME_INDEX_ERR = 0x30030,
+ ZXDH_TX_ACK_SYS_TOP_NVME_NOF_QID_ERR = 0x30031,
+ ZXDH_TX_ACK_SYS_TOP_NVME_NOF_PD_INDEX_ERR = 0x30032,
+ ZXDH_TX_ACK_SYS_TOP_NVME_LENGTH_ERR = 0x30033,
+ ZXDH_TX_ACK_SYS_TOP_NVME_KEY_ERR = 0x30034,
+ ZXDH_TX_ACK_SYS_TOP_NVME_ACCESS_ERR = 0x30035,
+ ZXDH_TX_ACK_SYS_TOP_MRTE_STATE_ERR = 0x50091,
+ ZXDH_TX_ACK_SYS_TOP_QP_CHECK_ERR = 0x50092,
+ ZXDH_TX_ACK_SYS_TOP_PD_CHECK_ERR = 0x50093,
+ ZXDH_TX_ACK_SYS_TOP_LKEY_CHECK_ERR = 0x50094,
+ ZXDH_TX_ACK_SYS_TOP_STAG_INDEX_CHECK_ERR = 0x50095,
+ ZXDH_TX_ACK_SYS_TOP_VADDR_LEN_CHECK_ERR = 0x50096,
+ ZXDH_TX_ACK_SYS_TOP_ACCESS_RIGHT_CHECK_ERR = 0x50097,
+ ZXDH_TX_ACK_SYS_TOP_STAG_INDEX_CHECK_ZERO_ERR = 0x50098,
+ ZXDH_TX_ACK_SYS_TOP_LOC_LEN_ERR = 0x600c0,
+ ZXDH_TX_ACK_SYS_TOP_NAK_INVALID_REQ = 0x700d0,
+ ZXDH_TX_ACK_SYS_TOP_NAK_REMOTE_ACCESS_ERR = 0x700d1,
+ ZXDH_TX_ACK_SYS_TOP_NAK_REMOTE_OPERATIONAL_ERR = 0x700d2,
+ ZXDH_TX_ACK_SYS_TOP_NAK_RETRY_LIMIT = 0x800f1,
+ ZXDH_TX_ACK_SYS_TOP_READ_RETRY_LIMIT = 0x800f2,
+ ZXDH_TX_ACK_SYS_TOP_TIMEOUT_RETRY_LIMIT = 0x800f3,
+ ZXDH_TX_ACK_SYS_TOP_RNR_RETRY_LIMIT = 0x800f4,
+};
+
+enum zxdh_rdmatx_window_top_err {
+ ZXDH_TX_WINDOW_TOP_WINDOW_NO_ENTRY = 0x800f5,
+ ZXDH_TX_WINDOW_TOP_WINDOW_BACK_MSN = 0x800f6,
+ ZXDH_TX_WINDOW_TOP_WINDOW_SMALL_MSN = 0x800f7,
+};
+
+enum zxdh_rdmatx_doorbell_mgr_err {
+ ZXDH_TX_DOORBELL_MGR_INDEX_CHECK_ERROR = 0x30036,
+ ZXDH_TX_DOORBELL_MGR_QID_CHECK_ERROR = 0x30037,
+ ZXDH_TX_DOORBELL_MGR_PD_INDEX_CHECK_ERROR = 0x30038,
+ ZXDH_TX_DOORBELL_MGR_LENGTH_CHECK_ERROR = 0x30039,
+ ZXDH_TX_DOORBELL_MGR_KEY_CHECK_ERROR = 0x3003a,
+ ZXDH_TX_DOORBELL_MGR_ACCESS_CHECK_ERROR = 0x3003b,
+};
+
+enum zxdh_rdmarx_err {
+ ZXDH_RX_CQP_FLUSH = 0x12,
+ ZXDH_RX_FIRST_PACKET_ERR = 0x4f,
+ ZXDH_RX_INVALID_OPCODE = 0x50,
+ ZXDH_RX_ORDER_ERR = 0x51,
+ ZXDH_RX_LEN_ERR = 0x52,
+ ZXDH_RX_SQR_STATE_ERR = 0x53,
+ ZXDH_RX_WQE_SIGN_ERR = 0x54,
+ ZXDH_RX_WQE_LEN_ERR = 0x55,
+ ZXDH_RX_SQR_WATER_LEVEL_ERR = 0x80,
+ ZXDH_RX_SRQ_AXI_RESP_ERR = 0xb1,
+ ZXDH_RX_CQ_OVERFLOW_ERR = 0x76,
+ ZXDH_RX_QP_CQ_OVERFLOW_ERR = 0x78,
+ ZXDH_RX_CQ_STATE_ERR = 0x7a,
+ ZXDH_RX_CQ_AXI_ERR = 0x7b,
+ ZXDH_RX_QP_CQ_AXI_ERR = 0x7c,
+ ZXDH_RX_NOF_IOQ_ERR = 0x70,
+ ZXDH_RX_NOF_PDNUM_ERR = 0x71,
+ ZXDH_RX_NOF_LEN_ERR = 0x72,
+ ZXDH_RX_NOF_RKEY_ERR = 0x73,
+ ZXDH_RX_NOF_ACC_ERR = 0x74,
+ ZXDH_RX_IRD_OVF = 0x77,
+ ZXDH_RX_MR_MW_STATE_FREE_ERR = 0x90,
+ ZXDH_RX_MR_MW_STATE_INVALID_ERR = 0x91,
+ ZXDH_RX_TYPE2B_MW_QPN_CHECK_ERR = 0x92,
+ ZXDH_RX_MR_MW_PD_CHECK_ERR = 0x93,
+ ZXDH_RX_MR_MW_KEY_CHECK_ERR = 0x94,
+ ZXDH_RX_MR_MW_STAG_INDEX_CHECK_ERR = 0x95,
+ ZXDH_RX_MR_MW_BOUNDARY_CHECK_ERR = 0x96,
+ ZXDH_RX_MR_MW_ACCESS_CHECK_ERR = 0x97,
+ ZXDH_RX_MR_MW_0STAG_INDEX_CHECK_ERR = 0x98,
+ ZXDH_RX_MW_STATE_INVALID_ERR = 0x99,
+ ZXDH_RX_MW_PD_CHECK_ERR = 0x9a,
+ ZXDH_RX_MW_RKEY_CHECK_ERR = 0x9b,
+ ZXDH_RX_TYPE2BMW_QPN_CHECK_ERR = 0x9c,
+ ZXDH_RX_MW_STAG_INDEX_CHECK_ERR = 0x9d,
+ ZXDH_RX_MW_SHARE_MR_CHECK_ERR = 0x9e,
+ ZXDH_RX_MW_TYPE1_CHECK_ERR = 0x9f,
+ ZXDH_RX_MR_PD_CHECK_ERR = 0xa0,
+ ZXDH_RX_MR_RKEY_CHECK_ERR = 0xa1,
+ ZXDH_RX_MR_SHARE_MR_CHECK_ERR = 0xa4,
+ ZXDH_RX_MR_BOND_MW_NUM_CHECK_ERR = 0xa5,
+ ZXDH_RX_MR_CANBE_R_INVALID_CHECK_ERR = 0xa6,
+ ZXDH_RX_AXI_RESP_ERR = 0xb0,
+};
+
struct zxdh_qp;
struct zxdh_cq;
struct zxdh_qp_init_info;
--
2.27.0

625
0051-libzrdma-Add-sq-rq-flush-cqe-and-log-optimization.patch Normal file
View File

@ -0,0 +1,625 @@
From 9f440ac471dce7fa44f0ecb6df4ca2ac0509d105 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E6=9D=8E=E5=AF=8C=E8=89=B3?= <li.fuyan@zte.com.cn>
Date: Fri, 28 Mar 2025 15:30:41 +0800
Subject: [PATCH] libzrdma:Add sq/rq flush cqe and log optimization
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: 李富艳 <li.fuyan@zte.com.cn>
---
providers/zrdma/main.h | 10 +-
providers/zrdma/zxdh_hw.c | 253 ++++++++++++++++++++++-------------
providers/zrdma/zxdh_verbs.c | 72 +++++-----
3 files changed, 206 insertions(+), 129 deletions(-)
diff --git a/providers/zrdma/main.h b/providers/zrdma/main.h
index e28c77b..7c78fed 100644
--- a/providers/zrdma/main.h
+++ b/providers/zrdma/main.h
@@ -38,15 +38,17 @@ enum {
ZXDH_DBG_SRQ = 1 << 2,
};
extern uint32_t zxdh_debug_mask;
-#define zxdh_dbg(ctx, mask, format, arg...) \
+#define zxdh_dbg(mask, format, arg...) \
do { \
if (mask & zxdh_debug_mask) { \
- int zxdh_dbg_tmp = errno; \
- verbs_debug(ctx, format, ##arg); \
- errno = zxdh_dbg_tmp; \
+ int tmp = errno; \
+ fprintf(stdout, "%s:%d: " format, __func__, __LINE__, \
+ ##arg); \
+ errno = tmp; \
} \
} while (0)
+
struct zxdh_udevice {
struct verbs_device ibv_dev;
};
diff --git a/providers/zrdma/zxdh_hw.c b/providers/zrdma/zxdh_hw.c
index 99489dc..fb8f016 100644
--- a/providers/zrdma/zxdh_hw.c
+++ b/providers/zrdma/zxdh_hw.c
@@ -1785,19 +1785,26 @@ static inline void build_comp_status(__u32 cq_type,
}
return;
}
- if (info->major_err == ZXDH_RETRY_ACK_MAJOR_ERR &&
- info->minor_err == ZXDH_RETRY_ACK_MINOR_ERR) {
- info->comp_status = ZXDH_COMPL_STATUS_RETRY_ACK_ERR;
+
+ switch (info->major_err) {
+ case ZXDH_RETRY_ACK_MAJOR_ERR:
+ if (info->minor_err == ZXDH_RETRY_ACK_MINOR_ERR) {
+ info->comp_status = ZXDH_COMPL_STATUS_RETRY_ACK_ERR;
+ return;
+ }
+ if (info->minor_err == ZXDH_TX_WINDOW_QUERY_ITEM_MINOR_ERR) {
+ info->comp_status =
+ ZXDH_COMPL_STATUS_TX_WINDOW_QUERY_ITEM_ERR;
+ return;
+ }
+ break;
+ case ZXDH_FLUSH_MAJOR_ERR:
+ info->comp_status = ZXDH_COMPL_STATUS_FLUSHED;
return;
- }
- if (info->major_err == ZXDH_RETRY_ACK_MAJOR_ERR &&
- info->minor_err == ZXDH_TX_WINDOW_QUERY_ITEM_MINOR_ERR) {
- info->comp_status = ZXDH_COMPL_STATUS_TX_WINDOW_QUERY_ITEM_ERR;
+ default:
+ info->comp_status = ZXDH_COMPL_STATUS_UNKNOWN;
return;
}
- info->comp_status = (info->major_err == ZXDH_FLUSH_MAJOR_ERR) ?
- ZXDH_COMPL_STATUS_FLUSHED :
- ZXDH_COMPL_STATUS_UNKNOWN;
}
__le64 *get_current_cqe(struct zxdh_cq *cq)
@@ -1837,9 +1844,9 @@ static inline void zxdh_get_cq_poll_info(struct zxdh_qp *qp,
}
}
-static void update_cq_poll_info(struct zxdh_qp *qp,
- struct zxdh_cq_poll_info *info, __u32 wqe_idx,
- __u64 qword0)
+static enum zxdh_status_code update_cq_poll_info(struct zxdh_qp *qp,
+ struct zxdh_cq_poll_info *info,
+ __u32 wqe_idx, __u64 qword0)
{
info->wr_id = qp->sq_wrtrk_array[wqe_idx].wrid;
if (!info->comp_status)
@@ -1847,6 +1854,7 @@ static void update_cq_poll_info(struct zxdh_qp *qp,
info->op_type = (__u8)FIELD_GET(ZXDHCQ_OP, qword0);
ZXDH_RING_SET_TAIL(qp->sq_ring,
wqe_idx + qp->sq_wrtrk_array[wqe_idx].quanta);
+ return ZXDH_SUCCESS;
}
static enum zxdh_status_code
@@ -1862,9 +1870,9 @@ process_tx_window_query_item_err(struct zxdh_qp *qp,
ib_qp = &iwuqp->vqp.qp;
ret = zxdh_query_qpc(ib_qp, &qpc);
if (ret) {
- verbs_err(verbs_get_ctx(ib_qp->context),
- "process tx window query item query qpc failed:%d\n",
- ret);
+ zxdh_dbg(ZXDH_DBG_QP,
+ "process tx window query item query qpc failed:%d\n",
+ ret);
return ZXDH_ERR_RETRY_ACK_ERR;
}
if (qpc.tx_last_ack_psn != qp->qp_last_ack_qsn)
@@ -1876,9 +1884,9 @@ process_tx_window_query_item_err(struct zxdh_qp *qp,
ret = zxdh_reset_qp(ib_qp, ZXDH_RESET_RETRY_TX_ITEM_FLAG);
if (ret) {
- verbs_err(verbs_get_ctx(ib_qp->context),
- "process tx window query item reset qp failed:%d\n",
- ret);
+ zxdh_dbg(ZXDH_DBG_QP,
+ "process tx window query item reset qp failed:%d\n",
+ ret);
return ZXDH_ERR_RETRY_ACK_ERR;
}
qp->qp_reset_cnt++;
@@ -1899,8 +1907,8 @@ process_retry_ack_err(struct zxdh_qp *qp, struct zxdh_cq_poll_info *info)
ib_qp = &iwuqp->vqp.qp;
ret = zxdh_query_qpc(ib_qp, &qpc);
if (ret) {
- verbs_err(verbs_get_ctx(ib_qp->context),
- "process retry ack query qpc failed:%d\n", ret);
+ zxdh_dbg(ZXDH_DBG_QP, "process retry ack query qpc failed:%d\n",
+ ret);
return ZXDH_ERR_RETRY_ACK_ERR;
}
if (!(qpc.retry_cqe_sq_opcode >= ZXDH_RETRY_CQE_SQ_OPCODE_ERR &&
@@ -1926,14 +1934,122 @@ process_retry_ack_err(struct zxdh_qp *qp, struct zxdh_cq_poll_info *info)
ZXDH_RETRY_CQE_SQ_OPCODE |
ZXDH_TX_READ_RETRY_FLAG_SET);
if (ret) {
- verbs_err(verbs_get_ctx(ib_qp->context),
- "process retry ack modify qpc failed:%d\n", ret);
+ zxdh_dbg(ZXDH_DBG_QP,
+ "process retry ack modify qpc failed:%d\n", ret);
return ZXDH_ERR_RETRY_ACK_ERR;
}
qp->cqe_retry_cnt++;
return ZXDH_ERR_RETRY_ACK_NOT_EXCEED_ERR;
}
+static enum zxdh_status_code
+zxdh_flush_sq_comp_info(struct zxdh_qp *qp, struct zxdh_cq_poll_info *info,
+ bool *move_cq_head)
+{
+ if (!ZXDH_RING_MORE_WORK(qp->sq_ring)) {
+ ZXDH_RING_INIT(qp->sq_ring, qp->sq_ring.size)
+ return ZXDH_ERR_Q_EMPTY;
+ }
+ do {
+ __le64 *sw_wqe;
+ __u64 wqe_qword;
+ __u64 wqe_idx;
+ wqe_idx = qp->sq_ring.tail;
+ sw_wqe = qp->sq_base[wqe_idx].elem;
+ get_64bit_val(sw_wqe, 0, &wqe_qword);
+ info->op_type = (__u8)FIELD_GET(ZXDHQPSQ_OPCODE, wqe_qword);
+ ZXDH_RING_SET_TAIL(qp->sq_ring,
+ wqe_idx +
+ qp->sq_wrtrk_array[wqe_idx].quanta);
+
+ if (info->op_type != ZXDH_OP_TYPE_NOP) {
+ info->wr_id = qp->sq_wrtrk_array[wqe_idx].wrid;
+ break;
+ }
+ } while (1);
+ qp->sq_flush_seen = true;
+ if (!ZXDH_RING_MORE_WORK(qp->sq_ring)) {
+ qp->sq_flush_complete = true;
+ ZXDH_RING_INIT(qp->sq_ring, qp->sq_ring.size)
+ } else
+ *move_cq_head = false;
+ return ZXDH_SUCCESS;
+}
+
+static enum zxdh_status_code zxdh_sq_comp_info(struct zxdh_qp *qp,
+ struct zxdh_cq_poll_info *info,
+ __u32 wqe_idx, __u64 qword0,
+ bool *move_cq_head)
+{
+ enum zxdh_status_code status_code;
+ switch (info->comp_status) {
+ case ZXDH_COMPL_STATUS_SUCCESS:
+ case ZXDH_COMPL_STATUS_UNKNOWN:
+ break;
+ case ZXDH_COMPL_STATUS_RETRY_ACK_ERR:
+ if (qp->qp_type == ZXDH_QP_TYPE_ROCE_RC) {
+ status_code = process_retry_ack_err(qp, info);
+ return (status_code == ZXDH_ERR_RETRY_ACK_ERR) ?
+ update_cq_poll_info(qp, info, wqe_idx,
+ qword0) :
+ status_code;
+ }
+ break;
+ case ZXDH_COMPL_STATUS_TX_WINDOW_QUERY_ITEM_ERR:
+ if (qp->qp_type == ZXDH_QP_TYPE_ROCE_RC) {
+ status_code =
+ process_tx_window_query_item_err(qp, info);
+ return (status_code == ZXDH_ERR_RETRY_ACK_ERR) ?
+ update_cq_poll_info(qp, info, wqe_idx,
+ qword0) :
+ status_code;
+ }
+ break;
+ case ZXDH_COMPL_STATUS_FLUSHED:
+ return zxdh_flush_sq_comp_info(qp, info, move_cq_head);
+ default:
+ break;
+ }
+ return update_cq_poll_info(qp, info, wqe_idx, qword0);
+}
+
+static enum zxdh_status_code zxdh_rq_comp_info(struct zxdh_qp *qp,
+ struct zxdh_cq_poll_info *info,
+ __u32 wqe_idx, __u64 qword2,
+ __u64 qword3, bool *move_cq_head)
+{
+ struct zxdh_uqp *iwuqp = NULL;
+ struct zxdh_usrq *iwusrq = NULL;
+ struct zxdh_srq *srq = NULL;
+ if (qp->is_srq) {
+ iwuqp = container_of(qp, struct zxdh_uqp, qp);
+ iwusrq = iwuqp->srq;
+ srq = &iwusrq->srq;
+ zxdh_free_srq_wqe(srq, wqe_idx);
+ info->wr_id = srq->srq_wrid_array[wqe_idx];
+ zxdh_get_cq_poll_info(qp, info, qword2, qword3);
+ } else {
+ if (unlikely(info->comp_status == ZXDH_COMPL_STATUS_FLUSHED ||
+ info->comp_status == ZXDH_COMPL_STATUS_UNKNOWN)) {
+ if (!ZXDH_RING_MORE_WORK(qp->rq_ring)) {
+ return ZXDH_ERR_Q_EMPTY;
+ }
+ wqe_idx = qp->rq_ring.tail;
+ }
+ info->wr_id = qp->rq_wrid_array[wqe_idx];
+ zxdh_get_cq_poll_info(qp, info, qword2, qword3);
+ ZXDH_RING_SET_TAIL(qp->rq_ring, wqe_idx + 1);
+ if (info->comp_status == ZXDH_COMPL_STATUS_FLUSHED) {
+ qp->rq_flush_seen = true;
+ if (!ZXDH_RING_MORE_WORK(qp->rq_ring))
+ qp->rq_flush_complete = true;
+ else
+ *move_cq_head = false;
+ }
+ }
+ return ZXDH_SUCCESS;
+}
+
/**
* zxdh_cq_poll_cmpl - get cq completion info
* @cq: hw cq
@@ -1942,7 +2058,6 @@ process_retry_ack_err(struct zxdh_qp *qp, struct zxdh_cq_poll_info *info)
enum zxdh_status_code zxdh_cq_poll_cmpl(struct zxdh_cq *cq,
struct zxdh_cq_poll_info *info)
{
- enum zxdh_status_code status_code;
__u64 comp_ctx, qword0, qword2, qword3;
__le64 *cqe;
struct zxdh_qp *qp;
@@ -1951,9 +2066,6 @@ enum zxdh_status_code zxdh_cq_poll_cmpl(struct zxdh_cq *cq,
int ret_code;
bool move_cq_head = true;
__u8 polarity;
- struct zxdh_usrq *iwusrq = NULL;
- struct zxdh_srq *srq = NULL;
- struct zxdh_uqp *iwuqp;
cqe = get_current_cqe(cq);
@@ -1973,7 +2085,7 @@ enum zxdh_status_code zxdh_cq_poll_cmpl(struct zxdh_cq *cq,
ret_code = ZXDH_ERR_Q_DESTROYED;
goto exit;
}
- iwuqp = container_of(qp, struct zxdh_uqp, qp);
+
info->qp_handle = (zxdh_qp_handle)(unsigned long)qp;
q_type = (__u8)FIELD_GET(ZXDH_CQ_SQ, qword0);
info->solicited_event = (bool)FIELD_GET(ZXDHCQ_SOEVENT, qword0);
@@ -1993,74 +2105,19 @@ enum zxdh_status_code zxdh_cq_poll_cmpl(struct zxdh_cq *cq,
info->qp_id = (__u32)FIELD_GET(ZXDHCQ_QPID, qword2);
info->imm_valid = false;
-
- info->qp_handle = (zxdh_qp_handle)(unsigned long)qp;
switch (q_type) {
- case ZXDH_CQE_QTYPE_RQ:
- if (qp->is_srq) {
- iwusrq = iwuqp->srq;
- srq = &iwusrq->srq;
- zxdh_free_srq_wqe(srq, wqe_idx);
- info->wr_id = srq->srq_wrid_array[wqe_idx];
- zxdh_get_cq_poll_info(qp, info, qword2, qword3);
- } else {
- if (unlikely(info->comp_status ==
- ZXDH_COMPL_STATUS_FLUSHED ||
- info->comp_status ==
- ZXDH_COMPL_STATUS_UNKNOWN)) {
- if (!ZXDH_RING_MORE_WORK(qp->rq_ring)) {
- ret_code = ZXDH_ERR_Q_EMPTY;
- goto exit;
- }
- wqe_idx = qp->rq_ring.tail;
- }
- info->wr_id = qp->rq_wrid_array[wqe_idx];
- zxdh_get_cq_poll_info(qp, info, qword2, qword3);
- ZXDH_RING_SET_TAIL(qp->rq_ring, wqe_idx + 1);
- if (info->comp_status == ZXDH_COMPL_STATUS_FLUSHED) {
- qp->rq_flush_seen = true;
- if (!ZXDH_RING_MORE_WORK(qp->rq_ring))
- qp->rq_flush_complete = true;
- else
- move_cq_head = false;
- }
- pring = &qp->rq_ring;
- }
- ret_code = ZXDH_SUCCESS;
- break;
case ZXDH_CQE_QTYPE_SQ:
- if (info->comp_status == ZXDH_COMPL_STATUS_RETRY_ACK_ERR &&
- qp->qp_type == ZXDH_QP_TYPE_ROCE_RC) {
- status_code = process_retry_ack_err(qp, info);
- if (status_code == ZXDH_ERR_RETRY_ACK_ERR) {
- update_cq_poll_info(qp, info, wqe_idx, qword0);
- ret_code = ZXDH_SUCCESS;
- } else {
- ret_code = status_code;
- }
- } else if (info->comp_status ==
- ZXDH_COMPL_STATUS_TX_WINDOW_QUERY_ITEM_ERR &&
- qp->qp_type == ZXDH_QP_TYPE_ROCE_RC) {
- status_code =
- process_tx_window_query_item_err(qp, info);
- if (status_code == ZXDH_ERR_RETRY_ACK_ERR) {
- update_cq_poll_info(qp, info, wqe_idx, qword0);
- ret_code = ZXDH_SUCCESS;
- } else {
- ret_code = status_code;
- }
- } else if (info->comp_status == ZXDH_COMPL_STATUS_FLUSHED) {
- info->wr_id = qp->sq_wrtrk_array[wqe_idx].wrid;
- ZXDH_RING_INIT(qp->sq_ring, qp->sq_ring.size);
- ret_code = ZXDH_SUCCESS;
- } else {
- update_cq_poll_info(qp, info, wqe_idx, qword0);
- ret_code = ZXDH_SUCCESS;
- }
+ ret_code = zxdh_sq_comp_info(qp, info, wqe_idx, qword0,
+ &move_cq_head);
+ pring = &qp->sq_ring;
+ break;
+ case ZXDH_CQE_QTYPE_RQ:
+ ret_code = zxdh_rq_comp_info(qp, info, wqe_idx, qword2, qword3,
+ &move_cq_head);
+ pring = &qp->rq_ring;
break;
default:
- zxdh_dbg(verbs_get_ctx(iwuqp->vqp.qp.context), ZXDH_DBG_CQ,
- "zxdh get cqe type unknow!\n");
+ zxdh_dbg(ZXDH_DBG_CQ, "zxdh get cqe type unknow!\n");
ret_code = ZXDH_ERR_Q_DESTROYED;
break;
}
@@ -2538,6 +2595,16 @@ enum zxdh_status_code zxdh_srq_init(struct zxdh_srq *srq,
ZXDH_RING_INIT(srq->srq_list_ring, srq->srq_list_size);
srq->srq_ring.tail = srq->srq_size - 1;
srq->srq_list_polarity = 1;
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s srq_wqe_size_multiplier:%d srqshift:%d\n",
+ __func__, srq->srq_wqe_size_multiplier, srqshift);
+ zxdh_dbg(
+ ZXDH_DBG_SRQ,
+ "%s srq->srq_id:%d srq_base:0x%p srq_list_base:0x%p srq_db_base:0x%p\n",
+ __func__, srq->srq_id, srq->srq_base, srq->srq_list_base,
+ srq->srq_db_base);
+ zxdh_dbg(ZXDH_DBG_SRQ,
+ "%s srq->srq_id:%d srq_ring_size:%d srq->srq_list_size:%d\n",
+ __func__, srq->srq_id, srq_ring_size, srq->srq_list_size);
return 0;
}
@@ -2558,4 +2625,6 @@ void zxdh_free_srq_wqe(struct zxdh_srq *srq, int wqe_index)
set_64bit_val(wqe, 0, hdr);
pthread_spin_unlock(&iwusrq->lock);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s srq->srq_id:%d wqe_index:%d\n", __func__,
+ srq->srq_id, wqe_index);
}
diff --git a/providers/zrdma/zxdh_verbs.c b/providers/zrdma/zxdh_verbs.c
index f67f8c7..9cf1240 100644
--- a/providers/zrdma/zxdh_verbs.c
+++ b/providers/zrdma/zxdh_verbs.c
@@ -1485,13 +1485,12 @@ static struct ibv_qp *create_qp(struct ibv_context *ibv_ctx,
}
if (attr->cap.max_inline_data > dev_attrs->max_hw_inline) {
- zxdh_dbg(&iwvctx->ibv_ctx, ZXDH_DBG_QP,
- "max_inline_data over max_hw_inline\n");
+ zxdh_dbg(ZXDH_DBG_QP, "max_inline_data over max_hw_inline\n");
attr->cap.max_inline_data = dev_attrs->max_hw_inline;
}
- zxdh_get_sq_wqe_shift(attr->cap.max_send_sge, attr->cap.max_inline_data,
- &sqshift);
+ zxdh_get_sq_wqe_shift(attr->cap.max_send_sge,
+ attr->cap.max_inline_data, &sqshift);
status = zxdh_get_sqdepth(dev_attrs, attr->cap.max_send_wr, sqshift,
&sqdepth);
if (status) {
@@ -2661,9 +2660,8 @@ static void zxdh_srq_wqe_init(struct zxdh_usrq *iwusrq)
__u64 hdr;
srq = &iwusrq->srq;
- zxdh_dbg(verbs_get_ctx(iwusrq->ibv_srq.context), ZXDH_DBG_SRQ,
- "%s head:%d tail:%d\n", __func__, srq->srq_ring.head,
- srq->srq_ring.tail);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s head:%d tail:%d\n", __func__,
+ srq->srq_ring.head, srq->srq_ring.tail);
for (i = srq->srq_ring.head; i < srq->srq_ring.tail; i++) {
wqe = zxdh_get_srq_wqe(srq, i);
@@ -2707,7 +2705,7 @@ static size_t zxdh_get_total_srq_size(struct zxdh_usrq *iwusrq, int srqdepth,
total_srq_queue_size + total_srq_list_size + total_srq_db_size;
iwusrq->total_buf_size = total_srq_size;
zxdh_dbg(
- verbs_get_ctx(iwusrq->ibv_srq.context), ZXDH_DBG_SRQ,
+ ZXDH_DBG_SRQ,
"%s total_srq_queue_size:%ld total_srq_list_size:%ld total_srq_db_size:%ld srqdepth:%d\n",
__func__, total_srq_queue_size, total_srq_list_size,
total_srq_db_size, srqdepth);
@@ -2730,7 +2728,7 @@ static int zxdh_alloc_srq_buf(struct zxdh_usrq *iwusrq,
(__le64 *)&info->srq_list_base[iwusrq->list_buf_size /
(sizeof(__u16))];
*(__le64 *)info->srq_db_base = ZXDH_SRQ_DB_INIT_VALUE;
- zxdh_dbg(verbs_get_ctx(iwusrq->ibv_srq.context), ZXDH_DBG_SRQ,
+ zxdh_dbg(ZXDH_DBG_SRQ,
"%s srq_base:0x%p srq_list_base:0x%p srq_db_base:0x%p\n",
__func__, info->srq_base, info->srq_list_base,
info->srq_db_base);
@@ -2782,7 +2780,7 @@ static int create_srq(struct ibv_pd *pd, struct zxdh_usrq *iwusrq,
info->srq_size = resp.actual_srq_size;
info->srq_list_size = resp.actual_srq_list_size;
zxdh_dbg(
- verbs_get_ctx(iwusrq->ibv_srq.context), ZXDH_DBG_SRQ,
+ ZXDH_DBG_SRQ,
"%s info->srq_id:%d info->srq_size:%d info->srq_list_size:%d\n",
__func__, info->srq_id, info->srq_size, info->srq_list_size);
@@ -2814,19 +2812,21 @@ static int zxdh_vmapped_srq(struct zxdh_usrq *iwusrq, struct ibv_pd *pd,
ret = zxdh_alloc_srq_buf(iwusrq, info, total_srq_size);
if (ret)
return -ENOMEM;
- zxdh_dbg(verbs_get_ctx(iwusrq->ibv_srq.context), ZXDH_DBG_SRQ,
- "%s srq_pages:%ld srq_list_pages:%ld\n", __func__, srq_pages,
- srq_list_pages);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s srq_pages:%ld srq_list_pages:%ld\n",
+ __func__, srq_pages, srq_list_pages);
ret = zxdh_reg_srq_mr(pd, info, total_srq_size, srq_pages,
srq_list_pages, iwusrq);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s %d ret:%d\n", __func__, __LINE__, ret);
if (ret) {
errno = ret;
goto err_dereg_srq_mr;
}
ret = create_srq(pd, iwusrq, attr, info);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s %d ret:%d\n", __func__, __LINE__, ret);
if (ret)
goto err_srq;
+
return 0;
err_srq:
ibv_cmd_dereg_mr(&iwusrq->vmr);
@@ -2907,8 +2907,7 @@ struct ibv_srq *zxdh_ucreate_srq(struct ibv_pd *pd,
dev_attrs = &iwvctx->dev_attrs;
if ((zxdh_check_srq_init_attr(srq_init_attr, dev_attrs)) != 0) {
- verbs_err(&iwvctx->ibv_ctx,
- "zxdh_check_srq_init_attr failed\n");
+ zxdh_dbg(ZXDH_DBG_SRQ, "zxdh_check_srq_init_attr failed\n");
errno = EINVAL;
return NULL;
}
@@ -2922,12 +2921,12 @@ struct ibv_srq *zxdh_ucreate_srq(struct ibv_pd *pd,
srq_init_attr->attr.max_wr, srqshift,
&srqdepth);
zxdh_dbg(
- &iwvctx->ibv_ctx, ZXDH_DBG_SRQ,
+ ZXDH_DBG_SRQ,
"%s %d status:%d srqshift:%d srqdepth:%d dev_attrs->max_hw_srq_quanta:%d srq_init_attr->attr.max_wr:%d\n",
__func__, __LINE__, status, srqshift, srqdepth,
dev_attrs->max_hw_srq_quanta, srq_init_attr->attr.max_wr);
if (status != 0) {
- verbs_err(&iwvctx->ibv_ctx, "zxdh_get_srqdepth failed\n");
+ zxdh_dbg(ZXDH_DBG_SRQ, "zxdh_get_srqdepth failed\n");
errno = EINVAL;
return NULL;
}
@@ -2940,19 +2939,21 @@ struct ibv_srq *zxdh_ucreate_srq(struct ibv_pd *pd,
if (zxdh_init_iwusrq(iwusrq, srq_init_attr, srqdepth, srqshift, &info,
dev_attrs)) {
- verbs_err(&iwvctx->ibv_ctx, "calloc srq_wrid_array failed\n");
+ zxdh_dbg(ZXDH_DBG_SRQ, "calloc srq_wrid_array failed\n");
goto err_srq_wrid_array;
}
status = zxdh_vmapped_srq(iwusrq, pd, srq_init_attr, srqdepth, &info);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s %d status:%d\n", __func__, __LINE__, status);
if (status) {
- verbs_err(&iwvctx->ibv_ctx, "zxdh_vmapped_srq failed\n");
+ zxdh_dbg(ZXDH_DBG_SRQ, "zxdh_vmapped_srq failed\n");
errno = status;
goto err_vmapped_srq;
}
status = zxdh_srq_init(&iwusrq->srq, &info);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s %d status:%d\n", __func__, __LINE__, status);
if (status) {
- verbs_err(&iwvctx->ibv_ctx, "zxdh_srq_init failed\n");
+ zxdh_dbg(ZXDH_DBG_SRQ, "zxdh_srq_init failed\n");
errno = EINVAL;
goto err_free_srq_init;
}
@@ -2960,9 +2961,8 @@ struct ibv_srq *zxdh_ucreate_srq(struct ibv_pd *pd,
srq_init_attr->attr.max_wr = (srqdepth - ZXDH_SRQ_RSVD) >> srqshift;
- zxdh_dbg(&iwvctx->ibv_ctx, ZXDH_DBG_SRQ,
- "iwusrq->srq_id:%d info.srq_size:%d\n", iwusrq->srq_id,
- info.srq_size);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s iwusrq->srq_id:%d info.srq_size:%d\n",
+ __func__, iwusrq->srq_id, info.srq_size);
return &iwusrq->ibv_srq;
err_free_srq_init:
@@ -2976,6 +2976,7 @@ err_srq_wrid_array:
errno = EINVAL;
err_free_srq:
free(iwusrq);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s %d\n", __func__, __LINE__);
return NULL;
}
@@ -2996,8 +2997,8 @@ int zxdh_udestroy_srq(struct ibv_srq *srq)
ret = zxdh_destroy_vmapped_srq(iwusrq);
if (ret)
goto err;
- zxdh_dbg(verbs_get_ctx(iwusrq->ibv_srq.context), ZXDH_DBG_SRQ,
- "iwusrq->srq_id:%d\n", iwusrq->srq_id);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s iwusrq->srq_id:%d\n", __func__,
+ iwusrq->srq_id);
zxdh_free_hw_buf(iwusrq->srq.srq_base, iwusrq->total_buf_size);
free(iwusrq->srq.srq_wrid_array);
free(iwusrq);
@@ -3024,9 +3025,8 @@ int zxdh_umodify_srq(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr,
sizeof(cmd));
if (ret == 0)
iwusrq->srq_limit = srq_attr->srq_limit;
- zxdh_dbg(verbs_get_ctx(iwusrq->ibv_srq.context), ZXDH_DBG_SRQ,
- "iwusrq->srq_id:%d srq_attr->srq_limit:%d\n", iwusrq->srq_id,
- srq_attr->srq_limit);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s iwusrq->srq_id:%d srq_attr->srq_limit:%d\n",
+ __func__, iwusrq->srq_id, srq_attr->srq_limit);
return ret;
}
@@ -3090,6 +3090,13 @@ static void zxdh_fill_srq_wqe(struct zxdh_usrq *iwusrq, struct zxdh_srq *srq,
udma_to_device_barrier(); /* make sure WQE is populated before valid bit is set */
set_64bit_val(wqe_64, 0, hdr);
+
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s wqe_64[0]:0x%llx\n", __func__, wqe_64[0]);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s wqe_64[1]:0x%llx\n", __func__, wqe_64[1]);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s wqe_64[2]:0x%llx\n", __func__, wqe_64[2]);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s wqe_64[3]:0x%llx\n", __func__, wqe_64[3]);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s wqe_64[4]:0x%llx\n", __func__, wqe_64[4]);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s wqe_64[5]:0x%llx\n", __func__, wqe_64[5]);
}
static void zxdh_get_wqe_index(struct zxdh_srq *srq, __le16 *wqe_16, __u16 *buf,
@@ -3112,6 +3119,7 @@ static void zxdh_update_srq_db_base(struct zxdh_usrq *iwusrq, __u16 idx)
udma_to_device_barrier(); /* make sure WQE is populated before valid bit is set */
set_64bit_val(iwusrq->srq.srq_db_base, 0, hdr);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s srq_db_base(hdr):0x%llx\n", __func__, hdr);
}
/**
@@ -3140,8 +3148,7 @@ int zxdh_upost_srq_recv(struct ibv_srq *srq, struct ibv_recv_wr *recv_wr,
buf_size = iwusrq->max_wr * sizeof(__u16);
buf = malloc(buf_size);
if (buf == NULL) {
- verbs_err(verbs_get_ctx(iwusrq->ibv_srq.context),
- "malloc buf_size failed\n");
+ zxdh_dbg(ZXDH_DBG_SRQ, "malloc buf_size failed\n");
err = -ENOMEM;
goto out;
}
@@ -3161,9 +3168,8 @@ int zxdh_upost_srq_recv(struct ibv_srq *srq, struct ibv_recv_wr *recv_wr,
zxdh_fill_srq_wqe(iwusrq, hw_srq, wqe_64, recv_wr);
}
- zxdh_dbg(verbs_get_ctx(iwusrq->ibv_srq.context), ZXDH_DBG_SRQ,
- "nreq:%d err:%d iwusrq->srq_id:%d\n", nreq, err,
- iwusrq->srq_id);
+ zxdh_dbg(ZXDH_DBG_SRQ, "%s nreq:%d err:%d iwusrq->srq_id:%d\n",
+ __func__, nreq, err, iwusrq->srq_id);
if (err == 0) {
zxdh_get_wqe_index(hw_srq, wqe_16, buf, nreq, &idx);
--
2.27.0

676
0052-libzrdma-Fix-capability-related-bugs.patch Normal file
View File

@ -0,0 +1,676 @@
From 2db3c164aea36d297eb3db7c54804037c2754c80 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E6=9D=8E=E5=AF=8C=E8=89=B3?= <li.fuyan@zte.com.cn>
Date: Fri, 28 Mar 2025 15:56:16 +0800
Subject: [PATCH] libzrdma:Fix capability related bugs
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: 李富艳 <li.fuyan@zte.com.cn>
---
providers/zrdma/zxdh_defs.h | 33 +-----
providers/zrdma/zxdh_hw.c | 47 ++++-----
providers/zrdma/zxdh_verbs.c | 196 +++++++++++++++++++++--------------
providers/zrdma/zxdh_verbs.h | 2 +-
4 files changed, 141 insertions(+), 137 deletions(-)
diff --git a/providers/zrdma/zxdh_defs.h b/providers/zrdma/zxdh_defs.h
index 8772e7b..ec0bebe 100644
--- a/providers/zrdma/zxdh_defs.h
+++ b/providers/zrdma/zxdh_defs.h
@@ -41,7 +41,7 @@
#define ZXDH_SQ_WQE_BYTESIZE 32
#define ZXDH_SRQ_WQE_MIN_SIZE 16
-#define ZXDH_SQ_RSVD 258
+#define ZXDH_SQ_RSVD 1
#define ZXDH_RQ_RSVD 1
#define ZXDH_SRQ_RSVD 1
@@ -252,29 +252,7 @@
(_retcode) = ZXDH_ERR_RING_FULL; \
} \
}
-#define ZXDH_SQ_RING_MOVE_HEAD(_ring, _retcode) \
- { \
- register __u32 size; \
- size = (_ring).size; \
- if (!ZXDH_SQ_RING_FULL_ERR(_ring)) { \
- (_ring).head = ((_ring).head + 1) % size; \
- (_retcode) = 0; \
- } else { \
- (_retcode) = ZXDH_ERR_RING_FULL; \
- } \
- }
-#define ZXDH_SQ_RING_MOVE_HEAD_BY_COUNT(_ring, _count, _retcode) \
- { \
- register __u32 size; \
- size = (_ring).size; \
- if ((ZXDH_RING_USED_QUANTA(_ring) + (_count)) < \
- (size - 256)) { \
- (_ring).head = ((_ring).head + (_count)) % size; \
- (_retcode) = 0; \
- } else { \
- (_retcode) = ZXDH_ERR_RING_FULL; \
- } \
- }
+
#define ZXDH_RING_MOVE_HEAD_BY_COUNT_NOCHECK(_ring, _count) \
(_ring).head = ((_ring).head + (_count)) % (_ring).size
@@ -298,13 +276,6 @@
#define ZXDH_ERR_RING_FULL3(_ring) \
((ZXDH_RING_USED_QUANTA(_ring) == ((_ring).size - 3)))
-#define ZXDH_SQ_RING_FULL_ERR(_ring) \
- ((ZXDH_RING_USED_QUANTA(_ring) == ((_ring).size - 257)))
-
-#define ZXDH_ERR_SQ_RING_FULL2(_ring) \
- ((ZXDH_RING_USED_QUANTA(_ring) == ((_ring).size - 258)))
-#define ZXDH_ERR_SQ_RING_FULL3(_ring) \
- ((ZXDH_RING_USED_QUANTA(_ring) == ((_ring).size - 259)))
#define ZXDH_RING_MORE_WORK(_ring) ((ZXDH_RING_USED_QUANTA(_ring) != 0))
#define ZXDH_RING_USED_QUANTA(_ring) \
diff --git a/providers/zrdma/zxdh_hw.c b/providers/zrdma/zxdh_hw.c
index fb8f016..0ea5a85 100644
--- a/providers/zrdma/zxdh_hw.c
+++ b/providers/zrdma/zxdh_hw.c
@@ -10,14 +10,6 @@
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
-#include <stdbool.h>
-#include <string.h>
-#include <util/udma_barrier.h>
-#include <util/util.h>
-#include <linux/types.h>
-#include <inttypes.h>
-#include <pthread.h>
-#include <endian.h>
#define ERROR_CODE_VALUE 65
static inline void qp_tx_psn_add(__u32 *x, __u32 y, __u16 mtu)
@@ -30,11 +22,17 @@ static inline void qp_tx_psn_add(__u32 *x, __u32 y, __u16 mtu)
*x = (*x + chunks) & 0xffffff;
}
-int zxdh_get_write_imm_split_switch(void)
+/**
+ * zxdh_fragcnt_to_quanta_sq - calculate quanta based on fragment count for SQ
+ * @frag_cnt: number of fragments
+ * @quanta: quanta for frag_cnt
+ */
+static inline enum zxdh_status_code zxdh_fragcnt_to_quanta_sq(__u32 frag_cnt, __u16 *quanta)
{
- char *env;
- env = getenv("ZXDH_WRITE_IMM_SPILT_ENABLE");
- return (env != NULL) ? atoi(env) : 0;
+ if (unlikely(frag_cnt > ZXDH_MAX_SQ_FRAG))
+ return ZXDH_ERR_INVALID_FRAG_COUNT;
+ *quanta = (frag_cnt >> 1) + 1;
+ return 0;
}
/**
@@ -2153,6 +2151,9 @@ int zxdh_qp_round_up(__u32 wqdepth)
{
int scount = 1;
+ if (wqdepth == 0)
+ return 0;
+
for (wqdepth--; scount <= 16; scount *= 2)
wqdepth |= wqdepth >> scount;
@@ -2167,6 +2168,9 @@ int zxdh_cq_round_up(__u32 wqdepth)
{
int scount = 1;
+ if (wqdepth == 0)
+ return 0;
+
for (wqdepth--; scount <= 16; scount *= 2)
wqdepth |= wqdepth >> scount;
@@ -2364,7 +2368,7 @@ enum zxdh_status_code zxdh_cq_init(struct zxdh_cq *cq,
void zxdh_clean_cq(void *q, struct zxdh_cq *cq)
{
__le64 *cqe;
- __u64 qword3, comp_ctx;
+ __u64 qword0, comp_ctx;
__u32 cq_head;
__u8 polarity, temp;
@@ -2377,8 +2381,8 @@ void zxdh_clean_cq(void *q, struct zxdh_cq *cq)
.buf;
else
cqe = cq->cq_base[cq_head].buf;
- get_64bit_val(cqe, 24, &qword3);
- polarity = (__u8)FIELD_GET(ZXDH_CQ_VALID, qword3);
+ get_64bit_val(cqe, 0, &qword0);
+ polarity = (__u8)FIELD_GET(ZXDH_CQ_VALID, qword0);
if (polarity != temp)
break;
@@ -2432,19 +2436,6 @@ enum zxdh_status_code zxdh_nop(struct zxdh_qp *qp, __u64 wr_id, bool signaled,
return 0;
}
-/**
- * zxdh_fragcnt_to_quanta_sq - calculate quanta based on fragment count for SQ
- * @frag_cnt: number of fragments
- * @quanta: quanta for frag_cnt
- */
-enum zxdh_status_code zxdh_fragcnt_to_quanta_sq(__u32 frag_cnt, __u16 *quanta)
-{
- if (frag_cnt > ZXDH_MAX_SQ_FRAG)
- return ZXDH_ERR_INVALID_FRAG_COUNT;
- *quanta = frag_cnt / 2 + 1;
- return 0;
-}
-
/**
* zxdh_fragcnt_to_wqesize_rq - calculate wqe size based on fragment count for RQ
* @frag_cnt: number of fragments
diff --git a/providers/zrdma/zxdh_verbs.c b/providers/zrdma/zxdh_verbs.c
index 9cf1240..39ff401 100644
--- a/providers/zrdma/zxdh_verbs.c
+++ b/providers/zrdma/zxdh_verbs.c
@@ -59,6 +59,7 @@ static int zxdh_get_inline_data(uint8_t *inline_data, struct ibv_send_wr *ib_wr,
while (num < ib_wr->num_sge) {
*len += ib_wr->sg_list[num].length;
if (*len > ZXDH_MAX_INLINE_DATA_SIZE) {
+ printf("err:inline bytes over max inline length\n");
return -EINVAL;
}
memcpy(inline_data + offset,
@@ -343,12 +344,8 @@ static void zxdh_free_hw_buf(void *buf, size_t size)
*/
static inline int get_cq_size(int ncqe)
{
- ncqe++;
-
- /* Completions with immediate require 1 extra entry */
if (ncqe < ZXDH_U_MINCQ_SIZE)
ncqe = ZXDH_U_MINCQ_SIZE;
-
return ncqe;
}
@@ -380,6 +377,7 @@ static struct ibv_cq_ex *ucreate_cq(struct ibv_context *context,
size_t total_size;
__u32 cq_pages;
int ret, ncqe;
+ __u64 resize_supported;
iwvctx = container_of(context, struct zxdh_uvcontext, ibv_ctx.context);
dev_attrs = &iwvctx->dev_attrs;
@@ -390,6 +388,13 @@ static struct ibv_cq_ex *ucreate_cq(struct ibv_context *context,
return NULL;
}
+ info.cq_size = get_cq_size(attr_ex->cqe);
+ info.cq_size = zxdh_cq_round_up(info.cq_size);
+ if (info.cq_size > dev_attrs->max_hw_cq_size) {
+ errno = EINVAL;
+ return NULL;
+ }
+
/* save the cqe requested by application */
ncqe = attr_ex->cqe;
iwucq = calloc(1, sizeof(*iwucq));
@@ -404,14 +409,13 @@ static struct ibv_cq_ex *ucreate_cq(struct ibv_context *context,
}
iwucq->resize_enable = false;
- info.cq_size = get_cq_size(attr_ex->cqe);
- info.cq_size = zxdh_cq_round_up(info.cq_size);
iwucq->comp_vector = attr_ex->comp_vector;
list_head_init(&iwucq->resize_list);
total_size = get_cq_total_bytes(info.cq_size);
cq_pages = total_size >> ZXDH_HW_PAGE_SHIFT;
+ resize_supported = dev_attrs->feature_flags & ZXDH_FEATURE_CQ_RESIZE;
- if (!(dev_attrs->feature_flags & ZXDH_FEATURE_CQ_RESIZE))
+ if (!resize_supported)
total_size = (cq_pages << ZXDH_HW_PAGE_SHIFT) +
ZXDH_DB_SHADOW_AREA_SIZE;
@@ -436,7 +440,7 @@ static struct ibv_cq_ex *ucreate_cq(struct ibv_context *context,
iwucq->vmr.ibv_mr.pd = &iwvctx->iwupd->ibv_pd;
- if (dev_attrs->feature_flags & ZXDH_FEATURE_CQ_RESIZE) {
+ if (resize_supported) {
info.shadow_area = zxdh_alloc_hw_buf(ZXDH_DB_SHADOW_AREA_SIZE);
if (!info.shadow_area)
goto err_dereg_mr;
@@ -457,7 +461,6 @@ static struct ibv_cq_ex *ucreate_cq(struct ibv_context *context,
}
iwucq->vmr_shadow_area.ibv_mr.pd = &iwvctx->iwupd->ibv_pd;
-
} else {
info.shadow_area = (__le64 *)((__u8 *)info.cq_base +
(cq_pages << ZXDH_HW_PAGE_SHIFT));
@@ -491,7 +494,9 @@ err_dereg_shadow:
ibv_cmd_dereg_mr(&iwucq->vmr);
if (iwucq->vmr_shadow_area.ibv_mr.handle) {
ibv_cmd_dereg_mr(&iwucq->vmr_shadow_area);
- zxdh_free_hw_buf(info.shadow_area, ZXDH_DB_SHADOW_AREA_SIZE);
+ if (resize_supported)
+ zxdh_free_hw_buf(info.shadow_area,
+ ZXDH_DB_SHADOW_AREA_SIZE);
}
err_dereg_mr:
zxdh_free_hw_buf(info.cq_base, total_size);
@@ -553,7 +558,7 @@ static int zxdh_process_resize_list(struct zxdh_ucq *iwucq,
struct zxdh_cq_buf *cq_buf, *next;
int cq_cnt = 0;
- list_for_each_safe (&iwucq->resize_list, cq_buf, next, list) {
+ list_for_each_safe(&iwucq->resize_list, cq_buf, next, list) {
if (cq_buf == lcqe_buf)
return cq_cnt;
@@ -774,7 +779,8 @@ static inline void zxdh_process_cqe(struct ibv_wc *entry,
*
* Returns the internal zxdh device error code or 0 on success
*/
-static int zxdh_poll_one(struct zxdh_cq *cq, struct zxdh_cq_poll_info *cur_cqe,
+static int zxdh_poll_one(struct zxdh_cq *cq,
+ struct zxdh_cq_poll_info *cur_cqe,
struct ibv_wc *entry)
{
int ret = zxdh_cq_poll_cmpl(cq, cur_cqe);
@@ -811,7 +817,7 @@ static int __zxdh_upoll_resize_cq(struct zxdh_ucq *iwucq, int num_entries,
int ret;
/* go through the list of previously resized CQ buffers */
- list_for_each_safe (&iwucq->resize_list, cq_buf, next, list) {
+ list_for_each_safe(&iwucq->resize_list, cq_buf, next, list) {
while (npolled < num_entries) {
ret = zxdh_poll_one(&cq_buf->cq, cur_cqe,
entry ? entry + npolled : NULL);
@@ -829,6 +835,7 @@ static int __zxdh_upoll_resize_cq(struct zxdh_ucq *iwucq, int num_entries,
cq_new_cqe = true;
continue;
}
+ printf("__zrdma_upoll_cq resize goto error failed\n");
goto error;
}
@@ -856,6 +863,7 @@ static int __zxdh_upoll_resize_cq(struct zxdh_ucq *iwucq, int num_entries,
cq_new_cqe = true;
continue;
}
+ printf("__zrdma_upoll_cq goto error failed\n");
goto error;
}
if (cq_new_cqe)
@@ -1038,7 +1046,7 @@ static uint64_t zxdh_wc_read_completion_wallclock_ns(struct ibv_cq_ex *ibvcq_ex)
container_of(ibvcq_ex, struct zxdh_ucq, verbs_cq.cq_ex);
/* RTT is in usec */
- return iwucq->cur_cqe.tcp_seq_num_rtt * 1000;
+ return (uint64_t)iwucq->cur_cqe.tcp_seq_num_rtt * 1000;
}
static enum ibv_wc_opcode zxdh_wc_read_opcode(struct ibv_cq_ex *ibvcq_ex)
@@ -1682,6 +1690,37 @@ int zxdh_uquery_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, int attr_mask,
sizeof(cmd));
}
+/**
+ * zxdh_clean_cqes - clean cq entries for qp
+ * @qp: qp for which completions are cleaned
+ * @iwcq: cq to be cleaned
+ */
+static void zxdh_clean_cqes(struct zxdh_qp *qp, struct zxdh_ucq *iwucq)
+{
+ struct zxdh_cq *ukcq = &iwucq->cq;
+ int ret;
+
+ ret = pthread_spin_lock(&iwucq->lock);
+ if (ret)
+ return;
+
+ zxdh_clean_cq(qp, ukcq);
+ pthread_spin_unlock(&iwucq->lock);
+}
+
+static void zxdh_init_qp_indices(struct zxdh_qp *qp)
+{
+ __u32 sq_ring_size;
+ sq_ring_size = ZXDH_RING_SIZE(qp->sq_ring);
+ ZXDH_RING_INIT(qp->sq_ring, sq_ring_size);
+ ZXDH_RING_INIT(qp->initial_ring, sq_ring_size);
+ qp->swqe_polarity = 0;
+ qp->swqe_polarity_deferred = 1;
+ qp->rwqe_polarity = 0;
+ qp->rwqe_signature = 0;
+ ZXDH_RING_INIT(qp->rq_ring, qp->rq_size);
+}
+
/**
* zxdh_umodify_qp - send qp modify to driver
* @qp: qp to modify
@@ -1705,6 +1744,18 @@ int zxdh_umodify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, int attr_mask)
} else {
ret = ibv_cmd_modify_qp(qp, attr, attr_mask, &cmd, sizeof(cmd));
}
+
+ if (!ret &&
+ (attr_mask & IBV_QP_STATE) &&
+ attr->qp_state == IBV_QPS_RESET) {
+ if (iwuqp->send_cq)
+ zxdh_clean_cqes(&iwuqp->qp, iwuqp->send_cq);
+
+ if (iwuqp->recv_cq && iwuqp->recv_cq != iwuqp->send_cq)
+ zxdh_clean_cqes(&iwuqp->qp, iwuqp->recv_cq);
+ zxdh_init_qp_indices(&iwuqp->qp);
+ }
+
if (!ret && (attr_mask & IBV_QP_PATH_MTU) &&
qp->qp_type == IBV_QPT_RC) {
mtu = mtu_enum_to_int(attr->path_mtu);
@@ -1736,24 +1787,6 @@ static void zxdh_issue_flush(struct ibv_qp *qp, bool sq_flush, bool rq_flush)
sizeof(cmd_ex), &resp, sizeof(resp));
}
-/**
- * zxdh_clean_cqes - clean cq entries for qp
- * @qp: qp for which completions are cleaned
- * @iwcq: cq to be cleaned
- */
-static void zxdh_clean_cqes(struct zxdh_qp *qp, struct zxdh_ucq *iwucq)
-{
- struct zxdh_cq *cq = &iwucq->cq;
- int ret;
-
- ret = pthread_spin_lock(&iwucq->lock);
- if (ret)
- return;
-
- zxdh_clean_cq(qp, cq);
- pthread_spin_unlock(&iwucq->lock);
-}
-
/**
* zxdh_udestroy_qp - destroy qp
* @qp: qp to destroy
@@ -1851,16 +1884,10 @@ int zxdh_upost_send(struct ibv_qp *ib_qp, struct ibv_send_wr *ib_wr,
struct zxdh_umr *umr = NULL;
__u64 mr_va = 0, mw_va = 0, value_dffer = 0, mw_pa_pble_index = 0;
__u16 mr_offset = 0;
- iwvctx = container_of(ib_qp->context, struct zxdh_uvcontext,
- ibv_ctx.context);
- if (ib_qp->state != IBV_QPS_RTS) {
- *bad_wr = ib_wr;
- verbs_err(&iwvctx->ibv_ctx, "zrdma: post send at state:%d\n",
- ib_qp->state);
- return -EINVAL;
- }
iwuqp = container_of(ib_qp, struct zxdh_uqp, vqp.qp);
+ iwvctx = container_of(ib_qp->context, struct zxdh_uvcontext,
+ ibv_ctx.context);
dev_attrs = &iwvctx->dev_attrs;
err = pthread_spin_lock(&iwuqp->lock);
@@ -1918,9 +1945,7 @@ int zxdh_upost_send(struct ibv_qp *ib_qp, struct ibv_send_wr *ib_wr,
iwuqp->inline_data, ib_wr,
&info.op.inline_rdma_send.len);
if (ret) {
- verbs_err(
- &iwvctx->ibv_ctx,
- "zrdma: get inline data fail\n");
+ printf("err:zxdh_get_inline_data fail\n");
pthread_spin_unlock(&iwuqp->lock);
return -EINVAL;
}
@@ -1937,11 +1962,11 @@ int zxdh_upost_send(struct ibv_qp *ib_qp, struct ibv_send_wr *ib_wr,
ib_wr->wr.ud.remote_qkey;
info.op.inline_rdma_send.dest_qp =
ib_wr->wr.ud.remote_qpn;
- ret = zxdh_ud_inline_send(&iwuqp->qp,
- &info, false);
+ ret = zxdh_ud_inline_send(
+ &iwuqp->qp, &info, false);
} else {
- ret = zxdh_rc_inline_send(&iwuqp->qp,
- &info, false);
+ ret = zxdh_rc_inline_send(
+ &iwuqp->qp, &info, false);
}
} else {
info.op.send.num_sges = ib_wr->num_sge;
@@ -1960,10 +1985,10 @@ int zxdh_upost_send(struct ibv_qp *ib_qp, struct ibv_send_wr *ib_wr,
info.op.inline_rdma_send.dest_qp =
ib_wr->wr.ud.remote_qpn;
ret = zxdh_ud_send(&iwuqp->qp, &info,
- false);
+ false);
} else {
ret = zxdh_rc_send(&iwuqp->qp, &info,
- false);
+ false);
}
}
if (ret)
@@ -1995,9 +2020,7 @@ int zxdh_upost_send(struct ibv_qp *ib_qp, struct ibv_send_wr *ib_wr,
iwuqp->inline_data, ib_wr,
&info.op.inline_rdma_write.len);
if (ret) {
- verbs_err(
- &iwvctx->ibv_ctx,
- "zrdma: get inline data fail\n");
+ printf("err:zxdh_get_inline_data fail\n");
pthread_spin_unlock(&iwuqp->lock);
return -EINVAL;
}
@@ -2007,8 +2030,8 @@ int zxdh_upost_send(struct ibv_qp *ib_qp, struct ibv_send_wr *ib_wr,
ib_wr->wr.rdma.remote_addr;
info.op.inline_rdma_write.rem_addr.stag =
ib_wr->wr.rdma.rkey;
- ret = zxdh_inline_rdma_write(&iwuqp->qp, &info,
- false);
+ ret = zxdh_inline_rdma_write(&iwuqp->qp,
+ &info, false);
} else {
info.op.rdma_write.lo_sg_list =
(void *)ib_wr->sg_list;
@@ -2017,7 +2040,8 @@ int zxdh_upost_send(struct ibv_qp *ib_qp, struct ibv_send_wr *ib_wr,
ib_wr->wr.rdma.remote_addr;
info.op.rdma_write.rem_addr.stag =
ib_wr->wr.rdma.rkey;
- ret = zxdh_rdma_write(&iwuqp->qp, &info, false);
+ ret = zxdh_rdma_write(&iwuqp->qp, &info,
+ false);
}
if (ret)
err = (ret == ZXDH_ERR_QP_TOOMANY_WRS_POSTED) ?
@@ -2036,7 +2060,8 @@ int zxdh_upost_send(struct ibv_qp *ib_qp, struct ibv_send_wr *ib_wr,
info.op.rdma_read.lo_sg_list = (void *)ib_wr->sg_list;
info.op.rdma_read.num_lo_sges = ib_wr->num_sge;
- ret = zxdh_rdma_read(&iwuqp->qp, &info, false, false);
+ ret = zxdh_rdma_read(&iwuqp->qp, &info, false,
+ false);
if (ret)
err = (ret == ZXDH_ERR_QP_TOOMANY_WRS_POSTED) ?
ENOMEM :
@@ -2383,20 +2408,17 @@ int zxdh_upost_send(struct ibv_qp *ib_qp, struct ibv_send_wr *ib_wr,
ret = zxdh_mw_bind(&iwuqp->qp, &info, false);
if (ret)
- err = (ret == ZXDH_ERR_QP_TOOMANY_WRS_POSTED) ?
- ENOMEM :
- EINVAL;
+ err = ENOMEM;
+
break;
case IBV_WR_LOCAL_INV:
info.op_type = ZXDH_OP_TYPE_LOCAL_INV;
info.op.inv_local_stag.target_stag =
ib_wr->invalidate_rkey;
ret = zxdh_stag_local_invalidate(&iwuqp->qp, &info,
- true);
+ true);
if (ret)
- err = (ret == ZXDH_ERR_QP_TOOMANY_WRS_POSTED) ?
- ENOMEM :
- EINVAL;
+ err = ENOMEM;
break;
default:
/* error */
@@ -2441,6 +2463,7 @@ int zxdh_upost_recv(struct ibv_qp *ib_qp, struct ibv_recv_wr *ib_wr,
if (unlikely(ib_qp->state == IBV_QPS_RESET || ib_qp->srq)) {
*bad_wr = ib_wr;
+ printf("err:post recv at reset or using srq\n");
return -EINVAL;
}
@@ -2490,9 +2513,18 @@ error:
struct ibv_ah *zxdh_ucreate_ah(struct ibv_pd *ibpd, struct ibv_ah_attr *attr)
{
struct zxdh_uah *ah;
+ union ibv_gid sgid;
struct zxdh_ucreate_ah_resp resp;
int err;
+ memset(&resp, 0, sizeof(resp));
+ err = ibv_query_gid(ibpd->context, attr->port_num, attr->grh.sgid_index,
+ &sgid);
+ if (err) {
+ errno = err;
+ return NULL;
+ }
+
ah = calloc(1, sizeof(*ah));
if (!ah)
return NULL;
@@ -2584,10 +2616,10 @@ int zxdh_uresize_cq(struct ibv_cq *cq, int cqe)
if (!(dev_attrs->feature_flags & ZXDH_FEATURE_CQ_RESIZE))
return -EOPNOTSUPP;
- if (cqe > ZXDH_MAX_CQ_SIZE)
+ if (cqe > dev_attrs->max_hw_cq_size)
return -EINVAL;
- cqe_needed = zxdh_cq_round_up(cqe + 1);
+ cqe_needed = zxdh_cq_round_up(cqe);
if (cqe_needed < ZXDH_U_MINCQ_SIZE)
cqe_needed = ZXDH_U_MINCQ_SIZE;
@@ -2609,6 +2641,10 @@ int zxdh_uresize_cq(struct ibv_cq *cq, int cqe)
goto err_buf;
}
+ ret = pthread_spin_lock(&iwucq->lock);
+ if (ret)
+ goto err_lock;
+
new_mr.ibv_mr.pd = iwucq->vmr.ibv_mr.pd;
reg_mr_cmd.reg_type = ZXDH_MEMREG_TYPE_CQ;
reg_mr_cmd.cq_pages = cq_pages;
@@ -2620,10 +2656,6 @@ int zxdh_uresize_cq(struct ibv_cq *cq, int cqe)
if (ret)
goto err_dereg_mr;
- ret = pthread_spin_lock(&iwucq->lock);
- if (ret)
- goto err_lock;
-
cmd.user_cq_buffer = (__u64)((uintptr_t)cq_base);
ret = ibv_cmd_resize_cq(&iwucq->verbs_cq.cq, cqe_needed, &cmd.ibv_cmd,
sizeof(cmd), &resp, sizeof(resp));
@@ -2642,10 +2674,10 @@ int zxdh_uresize_cq(struct ibv_cq *cq, int cqe)
return ret;
err_resize:
- pthread_spin_unlock(&iwucq->lock);
-err_lock:
ibv_cmd_dereg_mr(&new_mr);
err_dereg_mr:
+ pthread_spin_unlock(&iwucq->lock);
+err_lock:
free(cq_buf);
err_buf:
zxdh_free_hw_buf(cq_base, cq_size);
@@ -2735,7 +2767,8 @@ static int zxdh_alloc_srq_buf(struct zxdh_usrq *iwusrq,
return 0;
}
-static int zxdh_reg_srq_mr(struct ibv_pd *pd, struct zxdh_srq_init_info *info,
+static int zxdh_reg_srq_mr(struct ibv_pd *pd,
+ struct zxdh_srq_init_info *info,
size_t total_srq_size, uint16_t srq_pages,
uint16_t srq_list_pages, struct zxdh_usrq *iwusrq)
{
@@ -3043,7 +3076,8 @@ int zxdh_uquery_srq(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr)
}
static int zxdh_check_srq_valid(struct ibv_recv_wr *recv_wr,
- struct zxdh_usrq *iwusrq, struct zxdh_srq *srq)
+ struct zxdh_usrq *iwusrq,
+ struct zxdh_srq *srq)
{
if (unlikely(recv_wr->num_sge > iwusrq->max_sge))
return -EINVAL;
@@ -3054,8 +3088,9 @@ static int zxdh_check_srq_valid(struct ibv_recv_wr *recv_wr,
return 0;
}
-static void zxdh_fill_srq_wqe(struct zxdh_usrq *iwusrq, struct zxdh_srq *srq,
- __le64 *wqe_64, struct ibv_recv_wr *recv_wr)
+static void zxdh_fill_srq_wqe(struct zxdh_usrq *iwusrq,
+ struct zxdh_srq *srq, __le64 *wqe_64,
+ struct ibv_recv_wr *recv_wr)
{
__u32 byte_off;
int i;
@@ -3099,8 +3134,8 @@ static void zxdh_fill_srq_wqe(struct zxdh_usrq *iwusrq, struct zxdh_srq *srq,
zxdh_dbg(ZXDH_DBG_SRQ, "%s wqe_64[5]:0x%llx\n", __func__, wqe_64[5]);
}
-static void zxdh_get_wqe_index(struct zxdh_srq *srq, __le16 *wqe_16, __u16 *buf,
- __u16 nreq, __u16 *idx)
+static void zxdh_get_wqe_index(struct zxdh_srq *srq, __le16 *wqe_16,
+ __u16 *buf, __u16 nreq, __u16 *idx)
{
int i;
@@ -3207,3 +3242,10 @@ void zxdh_set_debug_mask(void)
if (env)
zxdh_debug_mask = strtol(env, NULL, 0);
}
+
+int zxdh_get_write_imm_split_switch(void)
+{
+ char *env;
+ env = getenv("ZXDH_WRITE_IMM_SPILT_ENABLE");
+ return (env != NULL) ? atoi(env) : 0;
+}
diff --git a/providers/zrdma/zxdh_verbs.h b/providers/zrdma/zxdh_verbs.h
index e3974c1..b72fa74 100644
--- a/providers/zrdma/zxdh_verbs.h
+++ b/providers/zrdma/zxdh_verbs.h
@@ -661,10 +661,10 @@ __le64 *zxdh_qp_get_next_recv_wqe(struct zxdh_qp *qp, __u32 *wqe_idx);
void zxdh_clean_cq(void *q, struct zxdh_cq *cq);
enum zxdh_status_code zxdh_nop(struct zxdh_qp *qp, __u64 wr_id, bool signaled,
bool post_sq);
-enum zxdh_status_code zxdh_fragcnt_to_quanta_sq(__u32 frag_cnt, __u16 *quanta);
enum zxdh_status_code zxdh_fragcnt_to_wqesize_rq(__u32 frag_cnt,
__u16 *wqe_size);
void zxdh_get_sq_wqe_shift(__u32 sge, __u32 inline_data, __u8 *shift);
+
void zxdh_get_rq_wqe_shift(__u32 sge, __u8 *shift);
enum zxdh_status_code zxdh_get_sqdepth(struct zxdh_dev_attrs *dev_attrs,
__u32 sq_size, __u8 shift,
--
2.27.0

152
0053-libhns-Clean-up-data-type-issues.patch Normal file
View File

@ -0,0 +1,152 @@
From 8f95635c359ca3c36f5b1b48889719b6840c07cc Mon Sep 17 00:00:00 2001
From: Junxian Huang <huangjunxian6@hisilicon.com>
Date: Thu, 13 Mar 2025 17:26:50 +0800
Subject: [PATCH 53/55] libhns: Clean up data type issues
mainline inclusion
from mainline-v56.0-65
commit fbe8827f270d0aff4a28bb645b826fa98fe00c9d
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IC1V44
CVE: NA
Reference: https://github.com/linux-rdma/rdma-core/pull/1579/commits/fbe8827f270d0aff4a...
---------------------------------------------------------------------
Clean up mixed signed/unsigned type issues. Fix a wrong format
character as well.
Fixes: cf6d9149f8f5 ("libhns: Introduce hns direct verbs")
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u.h | 4 ++--
providers/hns/hns_roce_u_hw_v2.c | 15 ++++++++-------
providers/hns/hns_roce_u_verbs.c | 6 +++---
3 files changed, 13 insertions(+), 12 deletions(-)
diff --git a/providers/hns/hns_roce_u.h b/providers/hns/hns_roce_u.h
index 5eedb81..e7e3f01 100644
--- a/providers/hns/hns_roce_u.h
+++ b/providers/hns/hns_roce_u.h
@@ -356,7 +356,7 @@ struct hns_roce_wq {
unsigned long *wrid;
struct hns_roce_spinlock hr_lock;
unsigned int wqe_cnt;
- int max_post;
+ unsigned int max_post;
unsigned int head;
unsigned int tail;
unsigned int max_gs;
@@ -392,7 +392,7 @@ struct hns_roce_qp {
struct verbs_qp verbs_qp;
struct hns_roce_buf buf;
struct hns_roce_dca_buf dca_wqe;
- int max_inline_data;
+ unsigned int max_inline_data;
unsigned int buf_size;
unsigned int sq_signal_bits;
struct hns_roce_wq sq;
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index 3137111..cea3043 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -173,7 +173,7 @@ static enum ibv_wc_status get_wc_status(uint8_t status)
{ HNS_ROCE_V2_CQE_XRC_VIOLATION_ERR, IBV_WC_REM_INV_RD_REQ_ERR },
};
- for (int i = 0; i < ARRAY_SIZE(map); i++) {
+ for (unsigned int i = 0; i < ARRAY_SIZE(map); i++) {
if (status == map[i].cqe_status)
return map[i].wc_status;
}
@@ -1189,7 +1189,7 @@ static int fill_ext_sge_inl_data(struct hns_roce_qp *qp,
unsigned int sge_mask = qp->ex_sge.sge_cnt - 1;
void *dst_addr, *src_addr, *tail_bound_addr;
uint32_t src_len, tail_len;
- int i;
+ uint32_t i;
if (sge_info->total_len > qp->sq.ext_sge_cnt * HNS_ROCE_SGE_SIZE)
return EINVAL;
@@ -1259,7 +1259,7 @@ static void fill_ud_inn_inl_data(const struct ibv_send_wr *wr,
static bool check_inl_data_len(struct hns_roce_qp *qp, unsigned int len)
{
- int mtu = mtu_enum_to_int(qp->path_mtu);
+ unsigned int mtu = mtu_enum_to_int(qp->path_mtu);
return (len <= qp->max_inline_data && len <= mtu);
}
@@ -1698,7 +1698,8 @@ static void fill_recv_sge_to_wqe(struct ibv_recv_wr *wr, void *wqe,
unsigned int max_sge, bool rsv)
{
struct hns_roce_v2_wqe_data_seg *dseg = wqe;
- unsigned int i, cnt;
+ unsigned int cnt;
+ int i;
for (i = 0, cnt = 0; i < wr->num_sge; i++) {
/* Skip zero-length sge */
@@ -1726,7 +1727,7 @@ static void fill_recv_inl_buf(struct hns_roce_rinl_buf *rinl_buf,
unsigned int wqe_idx, struct ibv_recv_wr *wr)
{
struct ibv_sge *sge_list;
- unsigned int i;
+ int i;
if (!rinl_buf->wqe_cnt)
return;
@@ -2053,7 +2054,7 @@ static int check_post_srq_valid(struct hns_roce_srq *srq,
static int get_wqe_idx(struct hns_roce_srq *srq, unsigned int *wqe_idx)
{
struct hns_roce_idx_que *idx_que = &srq->idx_que;
- int bit_num;
+ unsigned int bit_num;
int i;
/* bitmap[i] is set zero if all bits are allocated */
@@ -2451,7 +2452,7 @@ static void set_sgl_rc(struct hns_roce_v2_wqe_data_seg *dseg,
unsigned int mask = qp->ex_sge.sge_cnt - 1;
unsigned int msg_len = 0;
unsigned int cnt = 0;
- int i;
+ unsigned int i;
for (i = 0; i < num_sge; i++) {
if (!sge[i].length)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index 848f836..f0098ed 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -422,7 +422,7 @@ static int verify_cq_create_attr(struct ibv_cq_init_attr_ex *attr,
{
struct hns_roce_pad *pad = to_hr_pad(attr->parent_domain);
- if (!attr->cqe || attr->cqe > context->max_cqe) {
+ if (!attr->cqe || attr->cqe > (uint32_t)context->max_cqe) {
verbs_err(&context->ibv_ctx, "unsupported cq depth %u.\n",
attr->cqe);
return EINVAL;
@@ -1080,7 +1080,7 @@ static int check_hnsdv_qp_attr(struct hns_roce_context *ctx,
return 0;
if (!check_comp_mask(hns_attr->comp_mask, HNSDV_QP_SUP_COMP_MASK)) {
- verbs_err(&ctx->ibv_ctx, "invalid hnsdv comp_mask 0x%x.\n",
+ verbs_err(&ctx->ibv_ctx, "invalid hnsdv comp_mask 0x%llx.\n",
hns_attr->comp_mask);
return EINVAL;
}
@@ -1257,7 +1257,7 @@ static int alloc_recv_rinl_buf(uint32_t max_sge,
struct hns_roce_rinl_buf *rinl_buf)
{
unsigned int cnt;
- int i;
+ unsigned int i;
cnt = rinl_buf->wqe_cnt;
rinl_buf->wqe_list = calloc(cnt, sizeof(struct hns_roce_rinl_wqe));
--
2.33.0

63
0054-libhns-Fix-wrong-max-inline-data-value.patch Normal file
View File

@ -0,0 +1,63 @@
From 10534f0ef2ca73e8e59a38e51969cae864f9fbbf Mon Sep 17 00:00:00 2001
From: wenglianfa <wenglianfa@huawei.com>
Date: Thu, 13 Mar 2025 17:26:51 +0800
Subject: [PATCH 54/55] libhns: Fix wrong max inline data value
mainline inclusion
from mainline-v56.0-65
commit 8307b7c54ed81c343ec874e2066de79260b666d2
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IC1V44
CVE: NA
Reference: https://github.com/linux-rdma/rdma-core/pull/1579/commits/8307b7c54ed81c343e...
---------------------------------------------------------------------
When cap.max_inline_data is 0, it will be modified to 1 since
roundup_pow_of_two(0) == 1, which violates users' expectations.
Here fix it.
Fixes: 2aff0d55098c ("libhns: Fix the problem of sge nums")
Signed-off-by: wenglianfa <wenglianfa@huawei.com>
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u_verbs.c | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index f0098ed..5fe169e 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -1494,6 +1494,16 @@ static unsigned int get_sge_num_from_max_inl_data(bool is_ud,
return inline_sge;
}
+static uint32_t get_max_inline_data(struct hns_roce_context *ctx,
+ struct ibv_qp_cap *cap)
+{
+ if (cap->max_inline_data)
+ return min_t(uint32_t, roundup_pow_of_two(cap->max_inline_data),
+ ctx->max_inline_data);
+
+ return 0;
+}
+
static void set_ext_sge_param(struct hns_roce_context *ctx,
struct ibv_qp_init_attr_ex *attr,
struct hns_roce_qp *qp, unsigned int wr_cnt)
@@ -1510,9 +1520,7 @@ static void set_ext_sge_param(struct hns_roce_context *ctx,
attr->cap.max_send_sge);
if (ctx->config & HNS_ROCE_RSP_EXSGE_FLAGS) {
- attr->cap.max_inline_data = min_t(uint32_t, roundup_pow_of_two(
- attr->cap.max_inline_data),
- ctx->max_inline_data);
+ attr->cap.max_inline_data = get_max_inline_data(ctx, &attr->cap);
inline_ext_sge = max(ext_wqe_sge_cnt,
get_sge_num_from_max_inl_data(is_ud,
--
2.33.0

42
0055-libhns-Fix-wrong-order-of-spin-unlock-in-modify-qp.patch Normal file
View File

@ -0,0 +1,42 @@
From d1409106e1323c54fbbb0618c071efb024f58130 Mon Sep 17 00:00:00 2001
From: Junxian Huang <huangjunxian6@hisilicon.com>
Date: Thu, 13 Mar 2025 17:26:52 +0800
Subject: [PATCH 55/55] libhns: Fix wrong order of spin unlock in modify qp
mainline inclusion
from mainline-v56.0-65
commit d2b41c86c49335b3c6ab638abb1c0e31f5ba0e8f
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IC1V44
CVE: NA
Reference: https://github.com/linux-rdma/rdma-core/pull/1579/commits/d2b41c86c49335b3c6...
---------------------------------------------------------------------
The spin_unlock order should be the reverse of spin_lock order.
Fixes: 179f015e090d ("libhns: Add support for lock-free QP")
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u_hw_v2.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index cea3043..3a1249f 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -1910,8 +1910,8 @@ static int hns_roce_u_v2_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr,
if (flag) {
if (!ret)
qp->state = IBV_QPS_ERR;
- hns_roce_spin_unlock(&hr_qp->sq.hr_lock);
hns_roce_spin_unlock(&hr_qp->rq.hr_lock);
+ hns_roce_spin_unlock(&hr_qp->sq.hr_lock);
}
if (ret)
--
2.33.0

112
0056-libhns-Add-initial-support-for-HNS-LTTng-tracing.patch Normal file
View File

@ -0,0 +1,112 @@
From dfcef98e85b947dd38738436c769926f66438a7d Mon Sep 17 00:00:00 2001
From: wenglianfa <wenglianfa@huawei.com>
Date: Tue, 22 Apr 2025 16:18:44 +0800
Subject: [PATCH 56/57] libhns: Add initial support for HNS LTTng tracing
mainline inclusion
from mainline-v56.0-65
commit 5d96d96c822323a1c9b0a6b98ce58a17a8f165c1
category: feature
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IC3E67
CVE: NA
Reference: https://github.com/linux-rdma/rdma-core/pull/1587/commits/5d96d96c822323a1c9...
---------------------------------------------------------------------
Add initial support for HNS LTTng tracing.
Signed-off-by: wenglianfa <wenglianfa@huawei.com>
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/CMakeLists.txt | 10 +++++++++
providers/hns/hns_roce_u_trace.c | 9 ++++++++
providers/hns/hns_roce_u_trace.h | 35 ++++++++++++++++++++++++++++++++
3 files changed, 54 insertions(+)
create mode 100644 providers/hns/hns_roce_u_trace.c
create mode 100644 providers/hns/hns_roce_u_trace.h
diff --git a/providers/hns/CMakeLists.txt b/providers/hns/CMakeLists.txt
index 58139ae..36ebfac 100644
--- a/providers/hns/CMakeLists.txt
+++ b/providers/hns/CMakeLists.txt
@@ -1,5 +1,10 @@
+if (ENABLE_LTTNG AND LTTNGUST_FOUND)
+ set(TRACE_FILE hns_roce_u_trace.c)
+endif()
+
rdma_shared_provider(hns libhns.map
1 1.0.${PACKAGE_VERSION}
+ ${TRACE_FILE}
hns_roce_u.c
hns_roce_u_buf.c
hns_roce_u_db.c
@@ -12,3 +17,8 @@ publish_headers(infiniband
)
rdma_pkg_config("hns" "libibverbs" "${CMAKE_THREAD_LIBS_INIT}")
+
+if (ENABLE_LTTNG AND LTTNGUST_FOUND)
+ target_include_directories(hns PUBLIC ".")
+ target_link_libraries(hns LINK_PRIVATE LTTng::UST)
+endif()
diff --git a/providers/hns/hns_roce_u_trace.c b/providers/hns/hns_roce_u_trace.c
new file mode 100644
index 0000000..812f54c
--- /dev/null
+++ b/providers/hns/hns_roce_u_trace.c
@@ -0,0 +1,9 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Copyright (c) 2025 Hisilicon Limited.
+ */
+
+#define LTTNG_UST_TRACEPOINT_CREATE_PROBES
+#define LTTNG_UST_TRACEPOINT_DEFINE
+
+#include "hns_roce_u_trace.h"
diff --git a/providers/hns/hns_roce_u_trace.h b/providers/hns/hns_roce_u_trace.h
new file mode 100644
index 0000000..9b9485c
--- /dev/null
+++ b/providers/hns/hns_roce_u_trace.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * Copyright (c) 2025 Hisilicon Limited.
+ */
+
+#if defined(LTTNG_ENABLED)
+
+#undef LTTNG_UST_TRACEPOINT_PROVIDER
+#define LTTNG_UST_TRACEPOINT_PROVIDER rdma_core_hns
+
+#undef LTTNG_UST_TRACEPOINT_INCLUDE
+#define LTTNG_UST_TRACEPOINT_INCLUDE "hns_roce_u_trace.h"
+
+#if !defined(__HNS_TRACE_H__) || defined(LTTNG_UST_TRACEPOINT_HEADER_MULTI_READ)
+#define __HNS_TRACE_H__
+
+#include <lttng/tracepoint.h>
+#include <infiniband/verbs.h>
+
+#define rdma_tracepoint(arg...) lttng_ust_tracepoint(arg)
+
+#endif /* __HNS_TRACE_H__*/
+
+#include <lttng/tracepoint-event.h>
+
+#else
+
+#ifndef __HNS_TRACE_H__
+#define __HNS_TRACE_H__
+
+#define rdma_tracepoint(arg...)
+
+#endif /* __HNS_TRACE_H__*/
+
+#endif /* defined(LTTNG_ENABLED) */
--
2.33.0

382
0057-libhns-Add-tracepoint-for-HNS-RoCE-I-O.patch Normal file
View File

@ -0,0 +1,382 @@
From feec8deebf58cf6faaf9f70eda49b929eb674f72 Mon Sep 17 00:00:00 2001
From: wenglianfa <wenglianfa@huawei.com>
Date: Tue, 22 Apr 2025 16:18:45 +0800
Subject: [PATCH 57/57] libhns: Add tracepoint for HNS RoCE I/O
mainline inclusion
from mainline-v56.0-65
commit 19cb51c73029b593608f0c5d41a4ace8d1f1e334
category: feature
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IC3E67
CVE: NA
Reference: https://github.com/linux-rdma/rdma-core/pull/1587/commits/19cb51c73029b59360...
---------------------------------------------------------------------
Add tracepoint for HNS RoCE I/O, including post_send, post_recv and
poll_cq.
Signed-off-by: wenglianfa <wenglianfa@huawei.com>
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u_hw_v2.c | 153 +++++++++++++++++++++++++++++++
providers/hns/hns_roce_u_trace.h | 98 ++++++++++++++++++++
2 files changed, 251 insertions(+)
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index 3a1249f..b80c574 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -38,6 +38,7 @@
#include "hns_roce_u.h"
#include "hns_roce_u_db.h"
#include "hns_roce_u_hw_v2.h"
+#include "hns_roce_u_trace.h"
#define HR_IBV_OPC_MAP(ib_key, hr_key) \
[IBV_WR_ ## ib_key] = HNS_ROCE_WQE_OP_ ## hr_key
@@ -764,6 +765,80 @@ static int parse_cqe_for_cq(struct hns_roce_context *ctx, struct hns_roce_cq *cq
return 0;
}
+#ifdef LTTNG_ENABLED
+static uint8_t read_wc_sl(struct hns_roce_qp *hr_qp,
+ struct hns_roce_v2_cqe *cqe,
+ struct ibv_wc *wc)
+{
+ return hr_qp->verbs_qp.qp.qp_type == IBV_QPT_UD &&
+ hr_reg_read(cqe, CQE_S_R) == CQE_FOR_RQ ?
+ wc->sl : UINT8_MAX;
+}
+
+static uint32_t read_wc_rqpn(struct hns_roce_qp *hr_qp,
+ struct hns_roce_v2_cqe *cqe,
+ struct ibv_wc *wc)
+{
+ return hr_qp->verbs_qp.qp.qp_type == IBV_QPT_UD &&
+ hr_reg_read(cqe, CQE_S_R) == CQE_FOR_RQ ?
+ wc->src_qp : UINT32_MAX;
+}
+
+static uint32_t read_wc_byte_len(struct hns_roce_v2_cqe *cqe,
+ struct ibv_wc *wc)
+{
+ if (hr_reg_read(cqe, CQE_S_R) == CQE_FOR_RQ)
+ return wc->byte_len;
+
+ switch (hr_reg_read(cqe, CQE_OPCODE)) {
+ case HNS_ROCE_SQ_OP_RDMA_READ:
+ case HNS_ROCE_SQ_OP_ATOMIC_COMP_AND_SWAP:
+ case HNS_ROCE_SQ_OP_ATOMIC_FETCH_AND_ADD:
+ case HNS_ROCE_SQ_OP_ATOMIC_MASK_COMP_AND_SWAP:
+ case HNS_ROCE_SQ_OP_ATOMIC_MASK_FETCH_AND_ADD:
+ return wc->byte_len;
+ default:
+ return UINT32_MAX;
+ }
+}
+
+static uint8_t trace_wc_read_sl(struct ibv_cq_ex *cq_ex)
+{
+ return cq_ex->read_sl ? cq_ex->read_sl(cq_ex) : UINT8_MAX;
+}
+
+static uint32_t trace_wc_read_qp_num(struct ibv_cq_ex *cq_ex)
+{
+ return cq_ex->read_qp_num ?
+ cq_ex->read_qp_num(cq_ex) : UINT32_MAX;
+}
+
+static uint32_t trace_wc_read_src_qp(struct ibv_cq_ex *cq_ex)
+{
+ return cq_ex->read_src_qp ?
+ cq_ex->read_src_qp(cq_ex) : UINT32_MAX;
+}
+
+static uint32_t trace_wc_read_byte_len(struct ibv_cq_ex *cq_ex)
+{
+ return cq_ex->read_byte_len ?
+ cq_ex->read_byte_len(cq_ex) : UINT32_MAX;
+}
+
+static uint32_t get_send_wr_rqpn(struct ibv_send_wr *wr,
+ uint8_t qp_type)
+{
+ return qp_type == IBV_QPT_UD ? wr->wr.ud.remote_qpn : UINT32_MAX;
+}
+
+static uint8_t get_send_wr_tclass(struct ibv_send_wr *wr,
+ uint8_t qp_type)
+{
+ return qp_type == IBV_QPT_UD ?
+ to_hr_ah(wr->wr.ud.ah)->av.tclass : UINT8_MAX;
+}
+#endif
+
static int hns_roce_poll_one(struct hns_roce_context *ctx,
struct hns_roce_qp **cur_qp, struct hns_roce_cq *cq,
struct ibv_wc *wc)
@@ -800,8 +875,27 @@ static int hns_roce_poll_one(struct hns_roce_context *ctx,
wc->status = wc_status;
wc->vendor_err = hr_reg_read(cqe, CQE_SUB_STATUS);
wc->qp_num = qpn;
+
+ rdma_tracepoint(rdma_core_hns, poll_cq,
+ cq->verbs_cq.cq.context->device->name,
+ wc->wr_id, wc_status, wc->opcode,
+ wc->wc_flags, wc->vendor_err,
+ read_wc_sl(*cur_qp, cqe, wc),
+ wc->qp_num, read_wc_rqpn(*cur_qp, cqe, wc),
+ read_wc_byte_len(cqe, wc));
} else {
cq->verbs_cq.cq_ex.status = wc_status;
+
+ rdma_tracepoint(rdma_core_hns, poll_cq,
+ cq->verbs_cq.cq.context->device->name,
+ cq->verbs_cq.cq_ex.wr_id, wc_status,
+ ibv_wc_read_opcode(&cq->verbs_cq.cq_ex),
+ ibv_wc_read_wc_flags(&cq->verbs_cq.cq_ex),
+ ibv_wc_read_vendor_err(&cq->verbs_cq.cq_ex),
+ trace_wc_read_sl(&cq->verbs_cq.cq_ex),
+ trace_wc_read_qp_num(&cq->verbs_cq.cq_ex),
+ trace_wc_read_src_qp(&cq->verbs_cq.cq_ex),
+ trace_wc_read_byte_len(&cq->verbs_cq.cq_ex));
}
if (status == HNS_ROCE_V2_CQE_SUCCESS ||
@@ -1635,6 +1729,14 @@ int hns_roce_u_v2_post_send(struct ibv_qp *ibvqp, struct ibv_send_wr *wr,
*bad_wr = wr;
goto out;
}
+
+ rdma_tracepoint(rdma_core_hns, post_send,
+ ibvqp->context->device->name, wr->wr_id,
+ sge_info.valid_num, ibvqp->qp_num,
+ get_send_wr_rqpn(wr, ibvqp->qp_type),
+ wr->send_flags, sge_info.total_len,
+ wr->opcode, qp->sl,
+ get_send_wr_tclass(wr, ibvqp->qp_type));
}
out:
@@ -1785,6 +1887,10 @@ static int hns_roce_u_v2_post_recv(struct ibv_qp *ibvqp, struct ibv_recv_wr *wr,
wqe_idx = (qp->rq.head + nreq) & (qp->rq.wqe_cnt - 1);
fill_rq_wqe(qp, wr, wqe_idx, max_sge);
qp->rq.wrid[wqe_idx] = wr->wr_id;
+
+ rdma_tracepoint(rdma_core_hns, post_recv,
+ ibvqp->context->device->name, wr->wr_id,
+ wr->num_sge, ibvqp->qp_num, 0);
}
out:
@@ -2153,6 +2259,10 @@ static int hns_roce_u_v2_post_srq_recv(struct ibv_srq *ib_srq,
fill_wqe_idx(srq, wqe_idx);
srq->wrid[wqe_idx] = wr->wr_id;
+
+ rdma_tracepoint(rdma_core_hns, post_recv,
+ ib_srq->context->device->name, wr->wr_id,
+ wr->num_sge, srq->srqn, 1);
}
if (nreq) {
@@ -2442,6 +2552,12 @@ static void wr_set_sge_rc(struct ibv_qp_ex *ibv_qp, uint32_t lkey,
wqe->msg_len = htole32(length);
hr_reg_write(wqe, RCWQE_LEN0, length);
hr_reg_write(wqe, RCWQE_SGE_NUM, !!length);
+
+ rdma_tracepoint(rdma_core_hns, post_send,
+ ibv_qp->qp_base.context->device->name, ibv_qp->wr_id,
+ !!length, ibv_qp->qp_base.qp_num, UINT32_MAX,
+ ibv_qp->wr_flags, length,
+ hr_reg_read(wqe, RCWQE_OPCODE), qp->sl, UINT8_MAX);
}
static void set_sgl_rc(struct hns_roce_v2_wqe_data_seg *dseg,
@@ -2506,6 +2622,12 @@ static void wr_set_sge_list_rc(struct ibv_qp_ex *ibv_qp, size_t num_sge,
wqe->msg_len = htole32(qp->sge_info.total_len);
hr_reg_write(wqe, RCWQE_SGE_NUM, qp->sge_info.valid_num);
+
+ rdma_tracepoint(rdma_core_hns, post_send,
+ ibv_qp->qp_base.context->device->name, ibv_qp->wr_id,
+ qp->sge_info.valid_num, ibv_qp->qp_base.qp_num,
+ UINT32_MAX, ibv_qp->wr_flags, qp->sge_info.total_len,
+ opcode, qp->sl, UINT8_MAX);
}
static void wr_send_rc(struct ibv_qp_ex *ibv_qp)
@@ -2680,6 +2802,14 @@ static void set_inline_data_list_rc(struct hns_roce_qp *qp,
hr_reg_write(wqe, RCWQE_SGE_NUM, qp->sge_info.valid_num);
}
+
+ rdma_tracepoint(rdma_core_hns, post_send,
+ qp->verbs_qp.qp.context->device->name,
+ qp->verbs_qp.qp_ex.wr_id,
+ hr_reg_read(wqe, RCWQE_SGE_NUM),
+ qp->verbs_qp.qp.qp_num, UINT32_MAX,
+ qp->verbs_qp.qp_ex.wr_flags, msg_len,
+ hr_reg_read(wqe, RCWQE_OPCODE), qp->sl, UINT8_MAX);
}
static void wr_set_inline_data_rc(struct ibv_qp_ex *ibv_qp, void *addr,
@@ -2812,6 +2942,13 @@ static void wr_set_sge_ud(struct ibv_qp_ex *ibv_qp, uint32_t lkey,
dseg->len = htole32(length);
qp->sge_info.start_idx++;
+
+ rdma_tracepoint(rdma_core_hns, post_send,
+ ibv_qp->qp_base.context->device->name, ibv_qp->wr_id,
+ 1, ibv_qp->qp_base.qp_num,
+ hr_reg_read(wqe, UDWQE_DQPN), ibv_qp->wr_flags,
+ length, hr_reg_read(wqe, UDWQE_OPCODE),
+ qp->sl, hr_reg_read(wqe, UDWQE_TCLASS));
}
static void wr_set_sge_list_ud(struct ibv_qp_ex *ibv_qp, size_t num_sge,
@@ -2850,6 +2987,13 @@ static void wr_set_sge_list_ud(struct ibv_qp_ex *ibv_qp, size_t num_sge,
hr_reg_write(wqe, UDWQE_SGE_NUM, cnt);
qp->sge_info.start_idx += cnt;
+
+ rdma_tracepoint(rdma_core_hns, post_send,
+ ibv_qp->qp_base.context->device->name, ibv_qp->wr_id,
+ cnt, ibv_qp->qp_base.qp_num,
+ hr_reg_read(wqe, UDWQE_DQPN), ibv_qp->wr_flags,
+ msg_len, hr_reg_read(wqe, UDWQE_OPCODE),
+ qp->sl, hr_reg_read(wqe, UDWQE_TCLASS));
}
static void set_inline_data_list_ud(struct hns_roce_qp *qp,
@@ -2898,6 +3042,15 @@ static void set_inline_data_list_ud(struct hns_roce_qp *qp,
hr_reg_write(wqe, UDWQE_SGE_NUM, qp->sge_info.valid_num);
}
+
+ rdma_tracepoint(rdma_core_hns, post_send,
+ qp->verbs_qp.qp.context->device->name,
+ qp->verbs_qp.qp_ex.wr_id,
+ hr_reg_read(wqe, UDWQE_SGE_NUM),
+ qp->verbs_qp.qp.qp_num, hr_reg_read(wqe, UDWQE_DQPN),
+ qp->verbs_qp.qp_ex.wr_flags, msg_len,
+ hr_reg_read(wqe, UDWQE_OPCODE), qp->sl,
+ hr_reg_read(wqe, UDWQE_TCLASS));
}
static void wr_set_inline_data_ud(struct ibv_qp_ex *ibv_qp, void *addr,
diff --git a/providers/hns/hns_roce_u_trace.h b/providers/hns/hns_roce_u_trace.h
index 9b9485c..4654985 100644
--- a/providers/hns/hns_roce_u_trace.h
+++ b/providers/hns/hns_roce_u_trace.h
@@ -17,6 +17,104 @@
#include <lttng/tracepoint.h>
#include <infiniband/verbs.h>
+LTTNG_UST_TRACEPOINT_EVENT(
+ /* Tracepoint provider name */
+ rdma_core_hns,
+
+ /* Tracepoint name */
+ post_send,
+
+ /* Input arguments */
+ LTTNG_UST_TP_ARGS(
+ char *, dev_name,
+ uint64_t, wr_id,
+ int32_t, num_sge,
+ uint32_t, lqpn,
+ uint32_t, rqpn,
+ uint32_t, send_flags,
+ uint32_t, msg_len,
+ uint8_t, opcode,
+ uint8_t, sl,
+ uint8_t, t_class
+ ),
+
+ /* Output event fields */
+ LTTNG_UST_TP_FIELDS(
+ lttng_ust_field_string(dev_name, dev_name)
+ lttng_ust_field_integer_hex(uint64_t, wr_id, wr_id)
+ lttng_ust_field_integer_hex(int32_t, num_sge, num_sge)
+ lttng_ust_field_integer_hex(uint32_t, lqpn, lqpn)
+ lttng_ust_field_integer_hex(uint32_t, rqpn, rqpn)
+ lttng_ust_field_integer_hex(uint32_t, send_flags, send_flags)
+ lttng_ust_field_integer_hex(uint32_t, msg_len, msg_len)
+ lttng_ust_field_integer_hex(uint8_t, opcode, opcode)
+ lttng_ust_field_integer_hex(uint8_t, sl, sl)
+ lttng_ust_field_integer_hex(uint8_t, t_class, t_class)
+ )
+)
+
+LTTNG_UST_TRACEPOINT_EVENT(
+ /* Tracepoint provider name */
+ rdma_core_hns,
+
+ /* Tracepoint name */
+ post_recv,
+
+ /* Input arguments */
+ LTTNG_UST_TP_ARGS(
+ char *, dev_name,
+ uint64_t, wr_id,
+ int32_t, num_sge,
+ uint32_t, rqn,
+ uint8_t, is_srq
+ ),
+
+ /* Output event fields */
+ LTTNG_UST_TP_FIELDS(
+ lttng_ust_field_string(dev_name, dev_name)
+ lttng_ust_field_integer_hex(uint64_t, wr_id, wr_id)
+ lttng_ust_field_integer_hex(int32_t, num_sge, num_sge)
+ lttng_ust_field_integer_hex(uint32_t, rqn, rqn)
+ lttng_ust_field_integer_hex(uint8_t, is_srq, is_srq)
+ )
+)
+
+LTTNG_UST_TRACEPOINT_EVENT(
+ /* Tracepoint provider name */
+ rdma_core_hns,
+
+ /* Tracepoint name */
+ poll_cq,
+
+ /* Input arguments */
+ LTTNG_UST_TP_ARGS(
+ char *, dev_name,
+ uint64_t, wr_id,
+ uint8_t, status,
+ uint8_t, opcode,
+ uint8_t, wc_flags,
+ uint8_t, vendor_err,
+ uint8_t, pktype,
+ uint32_t, lqpn,
+ uint32_t, rqpn,
+ uint32_t, byte_len
+ ),
+
+ /* Output event fields */
+ LTTNG_UST_TP_FIELDS(
+ lttng_ust_field_string(dev_name, dev_name)
+ lttng_ust_field_integer_hex(uint64_t, wr_id, wr_id)
+ lttng_ust_field_integer_hex(uint8_t, status, status)
+ lttng_ust_field_integer_hex(uint8_t, opcode, opcode)
+ lttng_ust_field_integer_hex(uint8_t, wc_flags, wc_flags)
+ lttng_ust_field_integer_hex(uint8_t, vendor_err, vendor_err)
+ lttng_ust_field_integer_hex(uint8_t, pktype, pktype)
+ lttng_ust_field_integer_hex(uint32_t, lqpn, lqpn)
+ lttng_ust_field_integer_hex(uint32_t, rqpn, rqpn)
+ lttng_ust_field_integer_hex(uint32_t, byte_len, byte_len)
+ )
+)
+
#define rdma_tracepoint(arg...) lttng_ust_tracepoint(arg)
#endif /* __HNS_TRACE_H__*/
--
2.33.0

59
0058-libhns-Add-debug-log-for-lock-free-mode.patch Normal file
View File

@ -0,0 +1,59 @@
From 40c7b406829bc1250d93af527d70836e02c1fbac Mon Sep 17 00:00:00 2001
From: Junxian Huang <huangjunxian6@hisilicon.com>
Date: Thu, 24 Apr 2025 20:32:12 +0800
Subject: [PATCH 58/62] libhns: Add debug log for lock-free mode
mainline inclusion
from mainline-v56.0-65
commit fb96940fcf6f96185d407d57bcaf775ccf8f1762
category: cheanup
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IC3X57
CVE: NA
Reference:
https://github.com/linux-rdma/rdma-core/pull/1599/commits/fb96940fcf6f96185d...
---------------------------------------------------------------------
Currently there is no way to observe whether the lock-free mode is
configured from the driver's perspective. Add debug log for this.
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u_verbs.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index 5fe169e..3efc2f4 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -182,6 +182,7 @@ err:
struct ibv_pd *hns_roce_u_alloc_pad(struct ibv_context *context,
struct ibv_parent_domain_init_attr *attr)
{
+ struct hns_roce_pd *protection_domain;
struct hns_roce_pad *pad;
if (ibv_check_alloc_parent_domain(attr))
@@ -198,12 +199,16 @@ struct ibv_pd *hns_roce_u_alloc_pad(struct ibv_context *context,
return NULL;
}
+ protection_domain = to_hr_pd(attr->pd);
if (attr->td) {
pad->td = to_hr_td(attr->td);
atomic_fetch_add(&pad->td->refcount, 1);
+ verbs_debug(verbs_get_ctx(context),
+ "set PAD(0x%x) to lock-free mode.\n",
+ protection_domain->pdn);
}
- pad->pd.protection_domain = to_hr_pd(attr->pd);
+ pad->pd.protection_domain = protection_domain;
atomic_fetch_add(&pad->pd.protection_domain->refcount, 1);
atomic_init(&pad->pd.refcount, 1);
--
2.25.1

58
0059-libhns-Fix-ret-not-assigned-in-create-srq.patch Normal file
View File

@ -0,0 +1,58 @@
From 478e5fd1d8e1a0b04fe6638c163951a0892eab44 Mon Sep 17 00:00:00 2001
From: Junxian Huang <huangjunxian6@hisilicon.com>
Date: Wed, 23 Apr 2025 16:55:14 +0800
Subject: [PATCH 59/62] libhns: Fix ret not assigned in create srq()
mainline inclusion
from mainline-v56.0-65
commit 2034b1860c5a8b0cc3879315259462c04e53a98d
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IC3X57
CVE: NA
Reference:
https://github.com/linux-rdma/rdma-core/pull/1599/commits/2034b1860c5a8b0cc3...
---------------------------------------------------------------------
Fix the problem that ret may not be assigned in the error flow
of create_srq().
Fixes: aa7bcf7f7e44 ("libhns: Add support for lock-free SRQ")
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u_verbs.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index 3efc2f4..e0bafe3 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -933,16 +933,20 @@ static struct ibv_srq *create_srq(struct ibv_context *context,
if (pad)
atomic_fetch_add(&pad->pd.refcount, 1);
- if (hns_roce_srq_spinlock_init(context, srq, init_attr))
+ ret = hns_roce_srq_spinlock_init(context, srq, init_attr);
+ if (ret)
goto err_free_srq;
set_srq_param(context, srq, init_attr);
- if (alloc_srq_buf(srq))
+ ret = alloc_srq_buf(srq);
+ if (ret)
goto err_destroy_lock;
srq->rdb = hns_roce_alloc_db(hr_ctx, HNS_ROCE_SRQ_TYPE_DB);
- if (!srq->rdb)
+ if (!srq->rdb) {
+ ret = ENOMEM;
goto err_srq_buf;
+ }
ret = exec_srq_create_cmd(context, srq, init_attr);
if (ret)
--
2.25.1

99
0060-libhns-Fix-pad-refcnt-leaking-in-error-flow-of-creat.patch Normal file
View File

@ -0,0 +1,99 @@
From e45b9c648476b1b56592a873fa71699cb7f32ffd Mon Sep 17 00:00:00 2001
From: Junxian Huang <huangjunxian6@hisilicon.com>
Date: Wed, 23 Apr 2025 16:55:15 +0800
Subject: [PATCH 60/62] libhns: Fix pad refcnt leaking in error flow of create
qp/cq/srq
mainline inclusion
from mainline-v56.0-65
commit f877d6e610e438515e1535c9ec7a3a3ef37c58e0
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IC3X57
CVE: NA
Reference:
https://github.com/linux-rdma/rdma-core/pull/1599/commits/f877d6e610e438515e...
---------------------------------------------------------------------
Decrease pad refcnt by 1 in error flow of create qp/cq/srq.
Fixes: f8b4f622b1c5 ("libhns: Add support for lock-free QP")
Fixes: 95225025e24c ("libhns: Add support for lock-free CQ")
Fixes: aa7bcf7f7e44 ("libhns: Add support for lock-free SRQ")
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u_verbs.c | 20 +++++++++++++-------
1 file changed, 13 insertions(+), 7 deletions(-)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index e0bafe3..70f516a 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -445,12 +445,9 @@ static int verify_cq_create_attr(struct ibv_cq_init_attr_ex *attr,
return EOPNOTSUPP;
}
- if (attr->comp_mask & IBV_CQ_INIT_ATTR_MASK_PD) {
- if (!pad) {
- verbs_err(&context->ibv_ctx, "failed to check the pad of cq.\n");
- return EINVAL;
- }
- atomic_fetch_add(&pad->pd.refcount, 1);
+ if (attr->comp_mask & IBV_CQ_INIT_ATTR_MASK_PD && !pad) {
+ verbs_err(&context->ibv_ctx, "failed to check the pad of cq.\n");
+ return EINVAL;
}
attr->cqe = max_t(uint32_t, HNS_ROCE_MIN_CQE_NUM,
@@ -556,6 +553,7 @@ static void hns_roce_uninit_cq_swc(struct hns_roce_cq *cq)
static struct ibv_cq_ex *create_cq(struct ibv_context *context,
struct ibv_cq_init_attr_ex *attr)
{
+ struct hns_roce_pad *pad = to_hr_pad(attr->parent_domain);
struct hns_roce_context *hr_ctx = to_hr_ctx(context);
struct hns_roce_cq *cq;
int ret;
@@ -570,8 +568,10 @@ static struct ibv_cq_ex *create_cq(struct ibv_context *context,
goto err;
}
- if (attr->comp_mask & IBV_CQ_INIT_ATTR_MASK_PD)
+ if (attr->comp_mask & IBV_CQ_INIT_ATTR_MASK_PD) {
cq->parent_domain = attr->parent_domain;
+ atomic_fetch_add(&pad->pd.refcount, 1);
+ }
ret = hns_roce_cq_spinlock_init(context, cq, attr);
if (ret)
@@ -611,6 +611,8 @@ err_db:
err_buf:
hns_roce_spinlock_destroy(&cq->hr_lock);
err_lock:
+ if (attr->comp_mask & IBV_CQ_INIT_ATTR_MASK_PD)
+ atomic_fetch_sub(&pad->pd.refcount, 1);
free(cq);
err:
if (ret < 0)
@@ -977,6 +979,8 @@ err_destroy_lock:
hns_roce_spinlock_destroy(&srq->hr_lock);
err_free_srq:
+ if (pad)
+ atomic_fetch_sub(&pad->pd.refcount, 1);
free(srq);
err:
@@ -1872,6 +1876,8 @@ err_cmd:
err_buf:
hns_roce_qp_spinlock_destroy(qp);
err_spinlock:
+ if (pad)
+ atomic_fetch_sub(&pad->pd.refcount, 1);
free(qp);
err:
if (ret < 0)
--
2.25.1

69
0061-libhns-Fix-freeing-pad-without-checking-refcnt.patch Normal file
View File

@ -0,0 +1,69 @@
From 59108bf3e452fa7701a3972c78d22352598891be Mon Sep 17 00:00:00 2001
From: Junxian Huang <huangjunxian6@hisilicon.com>
Date: Wed, 23 Apr 2025 16:55:16 +0800
Subject: [PATCH 61/62] libhns: Fix freeing pad without checking refcnt
mainline inclusion
from mainline-v56.0-65
commit 234d135276ea8ef83633113e224e0cd735ebeca8
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IC3X57
CVE: NA
Reference:
https://github.com/linux-rdma/rdma-core/pull/1599/commits/234d135276ea8ef836...
---------------------------------------------------------------------
Currently pad refcnt will be added when creating qp/cq/srq, but it is
not checked when freeing pad. Add a check to prevent freeing pad when
it is still used by any qp/cq/srq.
Fixes: 7b6b3dae328f ("libhns: Add support for thread domain and parent
domain")
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
providers/hns/hns_roce_u_verbs.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index 70f516a..edd8e3d 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -218,14 +218,18 @@ struct ibv_pd *hns_roce_u_alloc_pad(struct ibv_context *context,
return &pad->pd.ibv_pd;
}
-static void hns_roce_free_pad(struct hns_roce_pad *pad)
+static int hns_roce_free_pad(struct hns_roce_pad *pad)
{
+ if (atomic_load(&pad->pd.refcount) > 1)
+ return EBUSY;
+
atomic_fetch_sub(&pad->pd.protection_domain->refcount, 1);
if (pad->td)
atomic_fetch_sub(&pad->td->refcount, 1);
free(pad);
+ return 0;
}
static int hns_roce_free_pd(struct hns_roce_pd *pd)
@@ -248,10 +252,8 @@ int hns_roce_u_dealloc_pd(struct ibv_pd *ibv_pd)
struct hns_roce_pad *pad = to_hr_pad(ibv_pd);
struct hns_roce_pd *pd = to_hr_pd(ibv_pd);
- if (pad) {
- hns_roce_free_pad(pad);
- return 0;
- }
+ if (pad)
+ return hns_roce_free_pad(pad);
return hns_roce_free_pd(pd);
}
--
2.25.1

43
0062-verbs-Assign-ibv-srq-pd-when-creating-SRQ.patch Normal file
View File

@ -0,0 +1,43 @@
From 387d76c0046b4fb6fbd8d70389b335661d099683 Mon Sep 17 00:00:00 2001
From: Junxian Huang <huangjunxian6@hisilicon.com>
Date: Wed, 23 Apr 2025 16:55:17 +0800
Subject: [PATCH 62/62] verbs: Assign ibv srq->pd when creating SRQ
mainline inclusion
from mainline-v56.0-65
commit bf1e427141fde2651bab4860e77a432bb7e26094
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IC3X57
CVE: NA
Reference:
https://github.com/linux-rdma/rdma-core/pull/1599/commits/bf1e427141fde2651b...
---------------------------------------------------------------------
Some providers need to access ibv_srq->pd during SRQ destruction, but
it may not be assigned currently when using ibv_create_srq_ex(). This
may lead to some SRQ-related resource leaks. Assign ibv_srq->pd when
creating SRQ to ensure pd can be obtained correctly.
Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com>
Signed-off-by: Xinghai Cen <cenxinghai@h-partners.com>
---
libibverbs/cmd_srq.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/libibverbs/cmd_srq.c b/libibverbs/cmd_srq.c
index dfaaa6a..259ea0d 100644
--- a/libibverbs/cmd_srq.c
+++ b/libibverbs/cmd_srq.c
@@ -63,6 +63,7 @@ static int ibv_icmd_create_srq(struct ibv_pd *pd, struct verbs_srq *vsrq,
struct verbs_xrcd *vxrcd = NULL;
enum ibv_srq_type srq_type;
+ srq->pd = pd;
srq->context = pd->context;
pthread_mutex_init(&srq->mutex, NULL);
pthread_cond_init(&srq->cond, NULL);
--
2.25.1

2742
0063-libxscale-update-to-version-2412GA.patch Normal file
View File

File diff suppressed because it is too large Load Diff

60
rdma-core.spec
View File

@ -1,6 +1,6 @@
Name: rdma-core
Version: 50.0
Release: 23
Release: 29
Summary: RDMA core userspace libraries and daemons
License: GPL-2.0-only OR BSD-2-Clause AND BSD-3-Clause
Url: https://github.com/linux-rdma/rdma-core
@ -51,6 +51,24 @@ patch42: 0042-libhns-Add-error-logs-to-help-diagnosis.patch
patch43: 0043-libhns-Fix-missing-fields-for-SRQ-WC.patch
patch44: 0044-libxscale-Add-Yunsilicon-User-Space-RDMA-Driver.patch
patch45: 0045-libxscale-Match-dev-by-vid-and-did.patch
patch46: 0046-libhns-fix-incorrectly-using-fixed-pagesize.patch
patch47: 0047-libhns-fix-missing-new-IO-support-for-DCA.patch
patch48: 0048-libzrdma-Fix-wqe-polarity-set-error.patch
patch49: 0049-libzrdma-Add-interface-aligned-with-kernel.patch
patch50: 0050-libzrdma-Add-poll-cqe-error-to-Failed-status.patch
patch51: 0051-libzrdma-Add-sq-rq-flush-cqe-and-log-optimization.patch
patch52: 0052-libzrdma-Fix-capability-related-bugs.patch
patch53: 0053-libhns-Clean-up-data-type-issues.patch
patch54: 0054-libhns-Fix-wrong-max-inline-data-value.patch
patch55: 0055-libhns-Fix-wrong-order-of-spin-unlock-in-modify-qp.patch
patch56: 0056-libhns-Add-initial-support-for-HNS-LTTng-tracing.patch
patch57: 0057-libhns-Add-tracepoint-for-HNS-RoCE-I-O.patch
patch58: 0058-libhns-Add-debug-log-for-lock-free-mode.patch
patch59: 0059-libhns-Fix-ret-not-assigned-in-create-srq.patch
patch60: 0060-libhns-Fix-pad-refcnt-leaking-in-error-flow-of-creat.patch
patch61: 0061-libhns-Fix-freeing-pad-without-checking-refcnt.patch
patch62: 0062-verbs-Assign-ibv-srq-pd-when-creating-SRQ.patch
patch63: 0063-libxscale-update-to-version-2412GA.patch
BuildRequires: binutils cmake >= 2.8.11 gcc libudev-devel pkgconfig pkgconfig(libnl-3.0)
BuildRequires: pkgconfig(libnl-route-3.0) systemd systemd-devel
@ -632,6 +650,42 @@ fi
%doc %{_docdir}/%{name}-%{version}/70-persistent-ipoib.rules
%changelog
* Thu May 8 2025 Xin Tian <tianx@yunsilicon.com> - 50.0-29
- Type: feature
- ID: NA
- SUG: NA
- DESC: [libxscale] update to version 2412GA
* Fri Apr 25 2025 Xinghai Cen <cenxinghai@h-partners.com> - 50.0-28
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: Bugfixes and one debug improvement
* Wed Apr 23 2025 Xinghai Cen <cenxinghai@h-partners.com> - 50.0-27
- Type: feature
- ID: NA
- SUG: NA
- DESC: libhns: Add support for LTTng tracing
* Thu Apr 17 2025 Xinghai Cen <cenxinghai@h-partners.com> - 50.0-26
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: libhns: Cleanup and Bugfixes
* Sat Mar 29 2025 Li Fuyan <li.fuyan@zte.com.cn> - 50.0-25
- Type: requirement
- ID: NA
- SUG: NA
- DESC: fix some libzrdma bugs and add some optimization
* Mon Mar 31 2025 Xinghai Cen <cenxinghai@h-partners.com> - 50.0-24
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: Fix some bugs for libhns
* Thu Mar 20 2025 Xin Tian <tianx@yunsilicon.com> - 50.0-23
- Type: bugfix
- ID: NA
@ -793,13 +847,13 @@ fi
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: Bugfix for lock and owner bit
- DESC: Bugfix for lock and owner bit
* Fri Dec 1 2023 Ran Zhou <zhouran10@h-partners.com> - 41.0-22
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: Bugfix for wrong timing of modifying ibv_qp state to err
- DESC: Bugfix for wrong timing of modifying ibv_qp state to err
* Mon Nov 27 2023 Ran Zhou <zhouran10@h-partners.com> - 41.0-21
- Type: bugfix