The Performance of 2023 American College of Rheumatology (ACR) / European Alliance of Associations for Rheumatology (EULAR) Antiphospholipid Syndrome Classification Criteria in a Real-World Rheumatology Department 

Ahmet Usta1, Müçteba Enes Yayla1, Emine Uslu1, Serdar Sezer1, Ebru Us2, Aşkın Ateş1 and Murat Turgay1.

1 Department of Rheumatology, Faculty of Medicine, Ankara University, Ankara, Turkey.
2 Department of Medical Microbiology, Faculty of Medicine, Ankara University, Ankara, Turkey.

.

Correspondence to: Müçteba Enes Yayla.  Department of Rheumatology, Ankara University Faculty of Medicine, No:82, Zip Code:06230 Sihhiye/Ankara/Turkey. Tel:+903125082234. E-mail: enesyayla@hotmail.com. ORCID: 0000-0002-5998-6703

Published: November 01, 2024
Received: May 26, 2024
Accepted: October 07, 2024
Mediterr J Hematol Infect Dis 2024, 16(1): e2024074 DOI 10.4084/MJHID.2024.074

This is an Open Access article distributed under the terms of the Creative Commons Attribution License
(https://creativecommons.org/licenses/by-nc/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: Antiphospholipid Syndrome (APS) is one of the most common acquired causes of hypercoagulability. The 2023 American College of Rheumatology (ACR) / European Alliance of Associations for Rheumatology (EULAR) APS Classification Criteria were specified as new APS classification criteria with high specificity for use in observational studies and research. The primary objective of this study was to evaluate the performance of the 2023 ACR/EULAR APS classification criteria in a real-world rheumatology department.
Methods: This is a retrospective, single-center study evaluating the sensitivity and specificity of the 2006 revised Sapporo and 2023 ACR/EULAR APS classification criteria in patients diagnosed with APS through clinical evaluation. A total of 184 patients, 103 of whom were diagnosed with APS, were included in the study.
Results: The 2023 ACR/EULAR APS classification criteria demonstrate higher specificity 98.8% (95% CI 93.3-99.8) and positive predictive value (PPV) 98.7% (95% CI 93.2-99.8). The revised Sapporo criteria exhibit higher sensitivity 90.3% (95% CI 83-96.6), negative predictive value (NPV) 88.1% (95% CI 79.4-93.4), and accuracy 90.8% (95% CI 85.7-94.1). When the diagnosis of APS was accepted according to the revised Sapporo criteria, the sensitivity of the 2023 ACR/EULAR APS classification criteria was 77% (95% CI 67.8-84.2), specificity 97.6% (95% CI 91.7-99.3), PPV 97.5% (95% CI 69.3-84.9) and NPV 78.1% (95% CI 69.3-84.9).
Conclusion: The 2023 ACR/EULAR APS classification criteria have low sensitivity and high specificity compared to the revised Sapporo APS classification criteria. The increase in specificity is due to risk assessment in thromboses and strict obstetric and laboratory criteria.

Introduction

Antiphospholipid Syndrome (APS) is one of the most common acquired causes of hypercoagulability. Clinical and laboratory diagnosis is characterized by the persistent presence of antiphospholipid antibodies (aPL) and venous and/or arterial thrombosis, pregnancy morbidity, and non-thrombotic clinical manifestations (e.g., heart valve thickening, livedo reticularis/racemosa). APS may be secondary to other systemic autoimmune diseases, such as systemic lupus erythematosus (SLE), or it may present as primary APS.[1,2]
The classification criteria for APS were first established as the Sapporo Criteria in 1999.[3] The Sapporo criteria were revised in 2006 after new clinical, laboratory, and experimental information became available. The revised criteria required the presence of at least one clinical and one laboratory criterion to classify APS.[4] Recently, the 2023 American College of Rheumatology (ACR) / European Alliance of Associations for Rheumatology (EULAR) APS Classification Criteria were specified as new APS classification criteria with high specificity for use in observational studies and research.[5] The 2023 ACR/EULAR classification criteria include [6] sets of clinical (macrovascular venous thromboembolism, macrovascular arterial thrombosis, microvascular, obstetric, valvular, and hematologic) and 2 sets of laboratory (lupus anticoagulant and anticardiolipin (aCL) IgG/IgM and/or anti-β2-glycoprotein I (aβ2GPI) IgG/IgM) criteria to be applied if at least one positive antiphospholipid antibody test within 3 years of the definition of an antiphospholipid antibody-related clinical criterion meets the entry criteria. Patients who score at least 3 points in each of the clinical and laboratory domains are classified as having APS.[5] It is hoped that these new criteria with high specificity will provide a solid basis for future APS research.
Clinicians diagnose APS based on clinical and laboratory evaluations. While, as in the case with other rheumatologic diseases, and as emphasized by publications on all criteria sets, classification criteria are not always applied in APS diagnosis. However, classification criteria provide a guide in clinical practice and have an important impact on real-life data. In this study, we aimed to measure the performance of the 2023 ACR/EULAR APS Classification Criteria and the Sapporo Classification Criteria revised in 2006 and to reveal their differences in patients with APS.

Materials and Methods

Participants. Between January 2010 and January 2024, a total of 103 patients who were being monitored at the Ankara University Faculty of Medicine Rheumatology Outpatient Clinic and diagnosed with APS through clinical evaluation by a rheumatologist were included in the study. A control group of 81 patients of a similar age and gender were selected. The members of the control group had sought consultation at the rheumatology clinic during the same period due to thrombosis/thromboembolism, pregnancy-related morbidities, or other clinical findings that suggested APS. However, they had not been diagnosed as such based on the clinical assessment of a rheumatologist. Patients with known hereditary thrombophilia and systemic vasculitis were excluded from the study. The study received approval from the Ethics Committee of the Ankara University Faculty of Medicine.
Data Collection and Classification. The clinical and laboratory data were examined retrospectively by reviewing digital patient records. The documentation included the existence of venous thromboembolism, arterial thrombosis, livedo racemosa, livedoid vasculopathy, APS nephropathy, pulmonary hemorrhage, cardiac illness, adrenal hemorrhage, and obstetric abnormalities. Serum aPL autoantibodies belonging to the IgG/IgM isotype were measured using an enzyme-linked immunoassay (Orgentec Diagnostika GmbH, Mainz, Germany). Highly purified cardiolipin is coated on microwells saturated with beta-2-glycoprotein I, and highly purified beta-2-glycoprotein I is bound to microwells. Integrated activated partial thromboplastin time (aPTT) test was used for lupus anticoagulant (LA) detection according to updated International Society on Hemostasis and Thrombosis (ISHT) guidelines.[6] The revised Sapporo APS criteria[4] and 2023 ACR/EULAR APS classification criteria[5] were tested on all participants.
Statistical Analysis. The SPSS version 21 software (SPSS, Chicago, USA) was used to analyze the data. For the purpose of comparing the characteristics of patients diagnosed with APS and the control group, the statistical analysis applied Pearson's chi-square or Fisher’s exact test to the categorical data and the Mann-Whitney U test to the quantitative data. A p-value below 0.05 was deemed to have statistical significance. Since there is no gold standard test for the diagnosis of APS, the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy values for classification tests were calculated with 95% confidence intervals (Wilson score method), both in those who were diagnosed with APS and those who were not diagnosed with APS by clinical evaluation.

Results

The median age of patients diagnosed with APS by clinical evaluation was 46.7 years (IQR 21.5), and 68.9% (71/103) were female. The median age of the control group was 46.5 years (IQR 23), and 59.3% (48/81) were female. There was no statistically significant difference between the two groups in terms of age (p = 0.867) and gender (p = 0.173). The demographic, clinical, and laboratory characteristics of all patients are presented in Table 1.
Table 1
Table 1. Comparison of demographic, clinical, and laboratory characteristics of patients with and without APS after clinical evaluation
Comparison of APS Classification Criteria. The 2023 ACR/EULAR APS classification criteria demonstrate higher specificity of 98.8% (95% CI 93.3-99.8) and positive predictive value (PPV) of 98.7% (95% CI 93.2-99.8), whereas the Revised Sapporo criteria exhibit higher sensitivity of 90.3% (95% CI 83-96.6), negative predictive value (NPV) of 88.1% (95% CI 79.4-93.4), and accuracy of 90.8% (95% CI 85.7-94.1) (Table 2). When the diagnosis of APS was accepted according to the Revised Sapporo criteria, the sensitivity of the 2023 ACR/EULAR APS classification criteria was 77% (95% CI 67.8-84.2), specificity was 97.6% (95% CI 91.7-99.3), PPV was 97.5% (95% CI 69.3-84.9) and NPV was 78.1% (95% CI 69.3-84.9) (Table 2).
Table 2
Table 2. Performance of APS Classification Criteria.
Patients Who Are Considered to Have APS but Do Not Meet at Least One APS Classification Criteria. Of the 103 patients who were accepted as APS after clinical evaluation, 8 (7.8%) did not meet either the 2023 ACR/EULAR APS or the revised Sapporo APS classification criteria (Supplemental Table S1). Two patients (1.9%) did meet the 2023 ACR/EULAR APS criteria but still need the revised Sapporo APS criteria. Both of these patients had thrombocytopenia, suspicion of aPL nephropathy, persistent LA positivity, moderate-high titer aCL IgG, and anti-β2GPI IgG. On the other hand, seventeen patients (16.5%) met the revised Sapporo APS criteria but not the 2023 ACR/EULAR APS criteria (Table 3).
Table 3
Table 3. Patients who meet Revised Sapporo APS Criteria but not 2023 ACR/EULAR APS Criteria.
Patients Who Were not Considered As Having APS but Did Meet at Least One APS Classification Criteria. As a result of clinical evaluation, seven patients (8.6%) who were not accepted for APS met the revised Sapporo APS classification. Of these seven patients, one also met the 2023 ACR/EULAR APS criteria (Table 4). Among the patients who were not accepted for APS, there were no patients who met the 2023 ACR/EULAR APS criteria but not the revised Sapporo APS.
Table 4
Table 4. Patients who do not considered APS but meet at least one classification criteria.

Discussion

In the present study, the 2023 ACR/EULAR APS classification criteria were shown to have high specificity and PPV in patients undergoing APS diagnostic evaluation by rheumatologists but lower sensitivity, NPV, and accuracy, when compared to the revised Sapporo APS classification criteria (Table 2). The ACR/EULAR APS Classification Criteria Collaborators aimed to achieve high specificity in the new 2023 ACR/EULAR APS classification criteria and in order to obtain a more homogeneous APS patient profile for research,[5] which in our case series supports the high specificity of the 2023 ACR/EULAR APS classification criteria.
Classification criteria aim to create more homogeneous cohorts to compare clinical studies rather than to make a diagnosis. Therefore, the classification criteria include more stringent standardized definitions. The 2023 ACR/EULAR APS classification criteria consist of different weighted criteria that assess the likelihood of APS in an individual, and these clinical and laboratory criteria increase the reliability and accuracy of the classification. In the study in which the criteria set was created, patients who met the 2006 revised Sapporo APS classification criteria were evaluated in the validation cohort, and sensitivity and specificity were found to be 84% and 99%, respectively.[5] There are very few studies evaluating the performance of 2023 ACR/EULAR APS classification criteria in APS cohorts. Zhao Y et al.[7] evaluated the performance of the 2023 ACR/EULAR APS classification criteria in APS patients diagnosed by expert rheumatologists and observed a sensitivity of 81.8% and a specificity of 98.3%. When Lu Q et al.[8] tested the new criteria in their APS cohort, they found a sensitivity of 78% and a specificity of 98%. Compared to the 2006 revised Sapporo APS classification criteria, the 2023 ACR/EULAR APS classification criteria appear to have high specificity and low sensitivity in the APS cohorts mentioned above. In our study, we evaluated the sensitivity and specificity of the 2023 APS classification criteria in two ways: firstly, in patients diagnosed with APS by a rheumatologist and secondly, in patients who met the revised Sapporo APS classification criteria. We found a sensitivity of 75.7%, a specificity of 98.8% in patients diagnosed with APS by clinical evaluation, a sensitivity of 77%, and a specificity of 97.6% in patients who met the revised Sapporo criteria. As in the studies mentioned above, when we compared the 2023 ACR/EULAR APS classification criteria with the Revised Sapporo APS classification criteria, we showed that they have high specificity and low sensitivity.
There were eight patients for whom we accepted APS based on clinical evaluation, but neither of the classification criteria was met (Supplemental Table S1). All eight patients had inadequate scores in terms of the revised Sapporo laboratory criteria. Furthermore, one patient also did not meet the clinical criteria. Naturally, patients who did not meet the revised Sapporo laboratory criteria did not have enough points for the laboratory domain in the 2023 ACR/EULAR APS classification criteria. Among these patients, three patients also did not have a sufficient score for the clinical domain. In Supplemental Table S1, Patient 8, who did not have thrombosis or pregnancy morbidity, was a patient whom we followed up together with the department of hematology with a diagnosis of APS due to thrombocytopenia, autoimmune hemolytic anemia, and low titer aCL IgM positivity. The characteristics of patients who met the revised Sapporo APS classification criteria but not the 2023 ACR/EULAR APS classification criteria are presented in Table 3. 13/17 (76.4%) patients did not meet just the clinical criteria, 2/17 (11.8%) patients did not meet just the laboratory criteria, and 2/17 (11.8%) patients did not meet either of the clinical and laboratory criteria. Having a high-risk profile for thromboembolism accounted for almost half of the reasons for not meeting the clinical criteria of the 2023 ACR/EULAR APS classification criteria. The Euro Phospholipid Project showed that in APS, 37.1% of patients were diagnosed with just venous thrombosis, 27% with just arterial thrombosis, 15% with arterial and venous thrombosis, and 8.6% with microvascular thrombosis.[9] An important reason for the low sensitivity of the 2023 ACR/EULAR APS Classification criteria is that the presence of thrombosis alone is not sufficient for clinical criteria. In the 2023 ACR/EULAR APS classification criteria, patients with macrovascular arterial and venous thrombosis are classified according to their risk profile for thrombosis. The presence of thrombosis in patients with a high-risk profile alone does not fulfill the clinical criteria requirement of the 2023 ACR/EULAR APS classification criteria.[5] Another reason for the low sensitivity is the strict scoring of obstetric criteria. At least three unexplained spontaneous abortions before 10 gestational weeks, early fetal loss (10 weeks-15 weeks 6 days), fetal death in the absence of severe pre-eclampsia (PEC) or severe placental insufficiency (PI) (16 weeks 0 days - 33 weeks 6 days) are sufficient for the 2006 revised Sapporo APS classification criteria. However, these pregnancy morbidities/mortalities alone are not sufficient to meet the 2023 ACR/EULAR APS classification criteria.[4,5] The low sensitivity and high specificity of the 2023 ACR/EULAR APS classification criteria in obstetric APS patients has recently been shown and is an expected result.[8] Barbhaiya et al.[10] state that recurrent pre-fetal deaths and early fetal deaths are not specific to APS, the most common causes being chromosomal abnormalities. It is also emphasized that fetal deaths without severe PEC or PI may be due to many non-APS causes. Therefore, these findings alone are not considered sufficient for the clinic in the new criteria.
Changes in laboratory criteria are another factor in the decrease in sensitivity. The detection of moderate or high titers of IgG and/or IgM aCL and/or aβ2GPI in serum determined by standard ELISA is sufficient for the 2006 revised Sapporo APS classification criteria. In the 2023 ACR/EULAR APS classification, only a single LAC positivity and/or the presence of medium-high titer aCL and/or aβ2GPI IgM-type antibody in the laboratory domain is insufficient. Persistent LAC positivity and/or the presence of medium-high titer aCL and/or aβ2GPI IgG-type antibody is required.[4,5] In addition, in APS patients, the association of IgA-type aPL antibodies with thrombosis is controversial, and they are not included in the 2023 ACR/EULAR APS and Revised Sapporo criteria due to a lack of sufficient data.[11] In our cohort, the main reason for inadequate laboratory criteria was the presence of IgM-type aPL. aPL antibodies can also be detected in the course of infectious diseases and are usually IgM-type aPL antibodies.[12] In addition, some drugs such as phenytoin, hydralazine, procainamide, and the presence of malignancy may lead to IgM type aPL positivity.[13-16] The presence of extensive thrombosis may affect activated partial thromboplastin time (aPTT) test in the acute phase and lead to false LAC positivity. In addition, anticoagulant use may also lead to the misevaluation of LAC antibodies.[17] In the new classification criteria, the power of aPL IgM-type antibodies and non-persistent LAC positivity in APS classification has been reduced. This change was an important contribution to the increase in the specificity of the new criteria. The factors mentioned above are the primary causes of the low sensitivity, but they are also the main reasons why the 2023 ACR/EULAR APS classification criteria have an extremely high specificity compared to the revised Sapporo criteria. This high specificity is a highly desirable performance characteristic of the new criteria for clinical trials and studies.
It is important to remember that the 2023 EULAR/APS classification criteria should not be employed for diagnostic purposes, and it is crucial to interpret the criteria accurately. The phrase "otherwise unexplained" is present in numerous domains in new criteria. For instance, thrombocytopenia that is caused by drug use or another factor should not be considered as being APS-related. It is also crucial to thoroughly assess the clinical findings that may be associated with systemic lupus erythematosus and to determine whether they are the result of APS or secondary to SLE.
The current study has certain limitations, such as a single-center retrospective study design, a lack of a gold-standard method for the diagnosis of APS, and utilizing a smaller number of patients compared to large APS cohorts. However, it also has the strength of being one of the few studies to evaluate the performance of the 2023 ACR/EULAR APS classification criteria on real-life data.

Conclusions

The 2023 ACR/EULAR APS classification criteria have low sensitivity and high specificity when compared to the revised Sapporo APS classification criteria. This increase in specificity is due to risk assessment in thromboses and strict obstetric and laboratory criteria.

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author upon reasonable request.

References   

  1. Garcia D, Erkan D. Diagnosis and Management of the Antiphospholipid Syndrome. N Engl J Med. 2018;378:2010-2021. https://doi.org/10.1056/NEJMra1705454
  2. Depietri L, Veropalumbo MR, Leone MC, Ghirarduzzi A. Antiphospholipid Syndrome: State of the Art of Clinical Management. Cardiovascular Drugs & Therapy. 2023. https://doi.org/10.1007/s10557-023-07496-3
  3. Wilson WA, Gharavi AE, Koike T, Lockshin MD, Branch DW, Piette JC, Brey R, Derksen R, Harris EN, Hughes GR, Triplett DA, Khamashta MA. International consensus statement on preliminary classification criteria for definite antiphospholipid syndrome: report of an international workshop. Arthritis & Rheumatism. 1999;42:1309-1311. https://doi.org/10.1002/1529-0131(199907)42:7<1309::AID-ANR1>3.0.CO;2-F
  4. Miyakis S, Lockshin MD, Atsumi T, Branch DW, Brey RL, Cervera R, Derksen RH, PG DEG, Koike T, Meroni PL, Reber G, Shoenfeld Y, Tincani A, Vlachoyiannopoulos PG, Krilis SA. International consensus statement on an update of the classification criteria for definite antiphospholipid syndrome (APS). Journal of Thrombosis & Haemostasis. 2006;4:295-306. https://doi.org/10.1111/j.1538-7836.2006.01753.x
  5. Barbhaiya M, Zuily S, Naden R, Hendry A, Manneville F, Amigo MC, Amoura Z, Andrade D, Andreoli L, Artim-Esen B, Atsumi T, Avcin T, Belmont HM, Bertolaccini ML, Branch DW, Carvalheiras G, Casini A, Cervera R, Cohen H, Costedoat-Chalumeau N, Crowther M, de Jesus G, Delluc A, Desai S, De Sancho M, Devreese KM, Diz-Kucukkaya R, Duarte-Garcia A, Frances C, Garcia D, Gris JC, Jordan N, Leaf RK, Kello N, Knight JS, Laskin C, Lee AI, Legault K, Levine SR, Levy RA, Limper M, Lockshin MD, Mayer-Pickel K, Musial J, Meroni PL, Orsolini G, Ortel TL, Pengo V, Petri M, Pons-Estel G, Gomez-Puerta JA, Raimboug Q, Roubey R, Sanna G, Seshan SV, Sciascia S, Tektonidou MG, Tincani A, Wahl D, Willis R, Yelnik C, Zuily C, Guillemin F, Costenbader K, Erkan D, Collaborators AEACC. The 2023 ACR/EULAR Antiphospholipid Syndrome Classification Criteria. Arthritis Rheumatol. 2023;75:1687-1702. https://doi.org/10.1002/art.42624
  6. Pengo V, Tripodi A, Reber G, Rand JH, Ortel TL, Galli M, De Groot PG, Subcommittee on Lupus Anticoagulant/Antiphospholipid Antibody of the S, Standardisation Committee of the International Society on T, Haemostasis. Update of the guidelines for lupus anticoagulant detection. Subcommittee on Lupus Anticoagulant/Antiphospholipid Antibody of the Scientific and Standardisation Committee of the International Society on Thrombosis and Haemostasis. Journal of Thrombosis & Haemostasis. 2009;7:1737-1740. https://doi.org/10.1111/j.1538-7836.2009.03555.x
  7. Zhao Y, Huang C, Zhou Y, Qi W, Cai B, Hu C, Song Y, Zhu T, Shi X, Liu X, Wang Q, Tian X, Zhao Y, Zeng X, Li M, Zhao J. Performance Validation of the 2023 ACR/EULAR Antiphospholipid Syndrome Classification Criteria in an Antiphospholipid Syndrome Cohort. Journal of Thrombosis & Haemostasis. 2024. https://doi.org/10.1136/annrheumdis-2024-eular.2006
  8. Lu Q, Gan Y, Yao Z, Li C. A diagnostic performance study of the 2023 American College of Rheumatology/European Alliance of Associations for Rheumatology classification criteria for patients with antiphospholipid syndrome from the Antiphospholipid Syndrome Chinese Collaborative cohort presenting with suspected antiphospholipid syndrome. Arthritis Rheumatol. 2024. https://doi.org/10.1002/art.42835
  9. Cervera R, Boffa M, Khamashta M, Hughes G. The Euro-Phospholipid project: epidemiology of the antiphospholipid syndrome in Europe. Lupus. 2009;18:889-893. https://doi.org/10.1177/0961203309106832
  10. Barbhaiya M, Zuily S, de Jesus G, Branch DW, Erkan D. Reply. Arthritis Rheumatol. 2024. https://doi.org/10.1002/art.42836
  11. Bertolaccini ML, Amengual O, Andreoli L, Atsumi T, Chighizola CB, Forastiero R, de Groot P, Lakos G, Lambert M, Meroni P, Ortel TL, Petri M, Rahman A, Roubey R, Sciascia S, Snyder M, Tebo AE, Tincani A, Willis R. 14th International Congress on Antiphospholipid Antibodies Task Force. Report on antiphospholipid syndrome laboratory diagnostics and trends. Autoimmun Rev. 2014;13:917-930. https://doi.org/10.1016/j.autrev.2014.05.001 
  12. Merrill JT, Asherson RA. Catastrophic antiphospholipid syndrome. Nat Clin Pract Rheumatol. 2006;2:81-89. https://doi.org/10.1038/ncprheum0069
  13. Vassalo J, Spector N, de Meis E, Rabello LS, Rosolem MM, do Brasil PE, Salluh JI, Soares M. Antiphospholipid antibodies in critically ill patients with cancer: a prospective cohort study. J Crit Care. 2014;29:533-538. https://doi.org/10.1016/j.jcrc.2014.02.005
  14. Triplett DA. Many faces of lupus anticoagulants. Lupus. 1998;7 Suppl 2:S18-22. https://doi.org/10.1177/096120339800700205
  15. Merrill JT, Shen C, Gugnani M, Lahita RG, Mongey AB. High prevalence of antiphospholipid antibodies in patients taking procainamide. J Rheumatol. 1997;24:1083-1088.
  16. Dlott JS, Roubey RA. Drug-induced lupus anticoagulants and antiphospholipid antibodies. Curr Rheumatol Rep. 2012;14:71-78. https://doi.org/10.1007/s11926-011-0227-1
  17. Favaloro EJ, Pasalic L. Lupus anticoagulant testing during anticoagulation, including direct oral anticoagulants. Res Pract Thromb Haemost. 2022;6:e12676. https://doi.org/10.1002/rth2.12676

Supplementary data
Supp Table S1
Supplemental Table S1. Patients who do not meet the 2023 ACR/EULAR APS Criteria and Revised Sapporo APS Criteria.