I have been working on causal inference and experimentation frameworks within the e-commerce tech industry lately, specifically dealing with platform behavioral and transaction telemetry data to determine how customers behaves on those e-commerce platforms.

Lately i have been facing problems where standard textbook statistical theory fails when dealing with messy, real-world data distributions as human behaviour is unpredictable and many unexpected things, which i didnt even thought could happen, can happen in the real world. Such as product delivery delay, causing the customers to be angry and hence put them off from buying the product, each of these variables have to be accounted for.

To model and have these unexpected psychological state among customers accounted for, we use variables such as sudden spikes in customer support chat volume, rapid page refreshes, or immediate session drops.

Also, Marketing headers and automated CRM systems love to push blanket promotional campaigns, but as statisticians, we know that measuring simple unadjusted A/B lift is a massive selection-bias trap.

I been trying to come up with a Statistical model that is trying to model complex human behavior—specifically isolating true customer revenue incrementality from dead-weight margin cannibalization. Essentially, trying to answer: Did the coupon cause the revenue spark, or did we just hand a discount to a power-buyer who was already determined to buy anyway even without the vouchers?

Are people in the tech industry genuinely valuing such skills nowadays? Using statistics to model and predict human behaviour and intent, answering questions like does our vouchers and ads really work?

I would love to hear your experiences how do u guys deals with the unpredictable messiness of human intent.

I’m considering taking a statistics course, and I’ve been thinking about the next steps, whether it’s significantly more beneficial for my future career to pursue a technical specialization, a master’s, or even a PhD

I am performing an ordinal regression analysis with 10 predictors, reduced from an original set of 50. Sample size: N = 380. One of these predictors is the primary predictor of interest (Safety Hazards [SH], a continuous variable). The rest are potential confounders of the primary predictor or contributors to the model.

My primary outcome is Job Satisfaction (JS).

I distilled the 10 candidates from the original 50 by running ordinal bivariate regressions on each candidate.

a) A crude model [JS <-- (beta)*SH]

b) 49 adjusted models: [JS <-- (beta)*SH + any of the 49 predictors].

1. If any of the bivariate regressions produced a CIE of 10% or more for the primary predictor, SH, it made the list of 10, regardless of its p- value.
2. If any of the 50 candidates had a p- value of < = 0.1, regardless of its CIE value, it made it to the list of 10 as well.

I sort the list of 10 by CIE value and use that order to determine which variable to test next in the model.

I then begin the final fitting process to extract the most plausible confounders of SH and true model contributors from the 10 predictors. I use 2 different regression methods: Forward Stepwise Regression (FSR) and Backward Stepwise Regression (BSR).

The conditions to stay in the model are: "If the p-value > 0.1 and CIE <10%, then reject from the model; otherwise, retain."

In FSR, I start with a sparse (crude) model, sequentially add a predictor, and test whether it meets the retention criteria. If so, I retain it, add the next predictor to the new model, and apply the same retention/rejection criteria until all predictors have been tested.

In BSR, I start with a full model and sequentially remove predictors, one at a time, and perform similar rejection/retention tests as in FSR until all predictors have been tested.

However, I find that the FSR and BSR algorithms retain and reject different variables. To be clear, about 60% of variables are common to both algorithms.

My question is: why do the FSR and BSR algorithms produce different results?

What is different about starting with a full model (BSR) versus a sparse model (FSR)?

What is different about starting with a sparse model (FSR) versus a full model (BSR)?

Please try to give an explanation that does not involve the term 'variance', as I tried to explain to my PI, only to be told that is the wrong way to explain it.

Thanks

David

Hi all, I picked up an interest in stats after undergrad & got to take it further in grad school. I took two levels of social science stats, advanced analytics & TA’d for undergrad stats.

I haven’t ruled out getting an MS in stats but I guess I still have some imposter syndrome about my fluency & whether I could succeed coming from social sciences. I’m in no rush to go back to grad school, so in the meantime I’d like to shore up my fundamentals & do some self-teaching. My question to this community is: what topics are must-know to be successful in a future Stats grad program & if you have any good free references to pass along (books/youtube playlists) I’d be very appreciative. Currently reading Bayesian Statistics: The Fun Way lol. Thank you!

Hello everyone! I hope this is an appropriate place to ask my question. I am a 2nd-year PharmD student who is currently in my first Evidence-Based Medicine class (aka journal club).

We are having an exam next week, and I really wanted to make sure to start brushing up on the things we've learned about so far. So, yesterday I was looking at some research studies on PubMed and trying to review key concepts.

One topic I have been trying to improve my knowledge of is effect size calculations, like Number Needed to Treat/Harm and Cohen's d.

I had thought I was getting a decent grasp on how to calculate these and when to use each one, but then I ran across a study where the primary outcome was the change in occurrences of a certain disease event (I will just refer to this disease event as "episodes," for brevity's sake) from baseline to the end of the study.

So, the results were posted as follows: The intervention treatment group had a change in occurrences of the episode of -25 (that is, they experienced an average of 25 fewer episodes than at baseline), while the control group had a change in episodes of -7. The study authors also included what appears to be an interquartile range alongside each result value.

I doubt my professor will put an article with this exact situation on the exam, but you never know, and since this is stumping me I would really like to try and understand what to do here. I don't think I could use Number Needed to Treat, since these results are not a simple yes/no categorical situation. But I don't know if I could do a Cohen's d, either, since the authors included an interquartile range but not a standard deviation, from what I can tell.

If anyone here in the Statistics subreddit community has any insight or advice about how to tackle the effect size calculations for a study like this, I would be very interested and appreciative to learn more about what to do when I run across this kind of data. Also, please note that NNT/NNH and Cohen's d are the only two effect size measurements this class has taught us, so if another type of calculation besides these ones would be more appropriate here, then it would be one that I have not heard of yet.

I want to study if one technique of teaching has a significant impact on learning. Yes, I could do a pre- and post-test and with a matched pair t-test and look for significance. But my problem is assuming that any teaching will result in some learning. My thought then is to teach in a traditional structure than again (different lesson) in the new method and check for significance between those two scores.

Or would it make sense to include both sets of tests and draw two conclusions

• Leads to significant learning (or not)
• Leads to significant learning (or not) over traditional methods

Hey, I have a decent background for distribution theory and statistical inference. I wanna get into random processes. I have no idea about the resources to start with. Can anyone suggest me some ?
Moreover, are there any discussion groups and channels on telegram ir any other platform specifically for this?

Hi everyone, just want to preface by saying i am very much a microbiologist and know extremely little in the world of stats. I’ll give some context to my experiment: I preformed an enrichment where i grew bacteria in vials (n= 3) for 6 weeks, then took a portion of those growths and added it into new vials (n=3) for another 6 weeks. The vials let’s say A1, B1, C1 corresponded to the second batch of vials A2, B2, C2 where in the samples from A1 was added to A2 etc. I did this at two different temperatures (20 and 30 degrees). So i have two variables, the time and the temperature, giving me four groups.
Group 1. 20 degrees celsius week 6 (rep1, rep2, rep3)
Group 2. 30 degrees celsius week 6 (rep1, rep2, rep3)
Group 3. 20 degrees celsius week 12 (rep1, rep2, rep3)
Group 4. 30 degrees celsius week 12 (rep1, rep2, rep3)

I have species level relative abundance data from shotgun metagenomic sequencing and want to determine whether community composition differs between the four groups and ideally a post hoc specific to which. The complication is that at each week 12 culture was subcultured from its corresponding week 6 culture with temperatures maintained. Therefore, week 6 and week 12 are linked observations rather than independent samples. There are three lineages per temperature. I’ve tried Bray curtis PERMANOVA in R using vegan::adonis2() with restricted permutations but the permutation schemes i’ve tried don’t test the temperature, week and interaction effects as intended. I also found significantly different multivariate dispersion between the temperature groups.

So my question is:
1. What statistical method would you recommend for testing community composition while accounting for the linked time points?
2. Is there an appropriate permutation scheme or alternative model for testing temperature, time, and their interaction with this design?
3. What methods would be suitable for identifying differentially abundant species from these metagenomic data?

I’m looking for a statistically defensible approach used in similar microbial enrichment or longitudinal studies. References to relevant papers or example R code would be greatly appreciated. Thank you!!!

Context: I have to send my final dissertation manuscript to my committee tomorrow. My chair called me this morning and told me my stats are all wrong because I ran too many tests with too small of a sample (10 participants). When I asked for guidance on resolving this, they said "You want a PhD? Figure it out". I have spent all morning reading resources and I have gotten myself more confused with each one.

This is a single group pre-post design based on feasibility and exploring preliminary clinical outcomes. I originally had planned paired samples t-tests, but I ended up with such a small sample that it would inflate error rate. Also because of the small sample, any assumptions of underlying distributions can't be met, so I'm using exact sign tests. I modeled my stats after another published pilot study with 7 participants and 15 comparisons. They do not do a correction.

I ran the 7 exact sign tests for my outcome variables, found significance in 6/7, and also presented the data visually. If I did a Bonferroni correction to control for multiple comparisons, my new p-value would be .007 and basically impossible to get with exact sign tests. Is my only option to only present stats for my main variable, and only present descriptive/visuals for the rest?

🎞Watch the short demo here: https://www.reddit.com/r/RStudio/s/K88EvqP6EC

Hello everyone! After 8 months (of rather sporadic) work, I am really excited to share the biggest update to SimtablR, an R package designed to streamline the most common analytical tasks in epidemiology and clinical research! (short demo video above)

I do epidemiological research in R, and I kept juggling multiple packages and repeating work just to get tables that were actually informative and ready for a paper. I also found that base R tabulation left a lot to be desired in terms of formatting, but gt_summary was a bit too slow and had a lot of boilerplate. This package is also great for people transitioning from Stata or still learning R!

With this new release, SimtablR now covers most of a typical analysis, from the first cross-table to the finished table in a Word document:

1. tb(): quick, but powerfull tabulation with formatting

• Row/col/total percentages with simple flags (row, col, cell) and adjustable decimals
• Hypothesis tests with one argument (N-1 chi-squared, Fisher when needed, etc.)
• Prevalence ratios, risk ratios or odds ratios with 95% CIs, directly in the table
• SimtablR now picks the right measure (RR, OR or PR) automatically for a stated study design (design = "cohort", "case_control", "cross_sectional")
• New: stratified analysis with Mantel-Haenszel pooled estimates

2. table1(): baseline tables of multiple variables (literally the "table 1")

• Automatically chooses mean (SD) or median (IQR) based on each variable’s distribution
• Reports missing data per variable and group
• Add standardized mean differences with test(smd = TRUE)
• Completely customizable and exportable to all office programs

3. diag_test() and roc(): diagnostic accuracy

• Automatically calculates the confusion matrix, sensitivity, specificity, PPV, NPV, LR+, LR− and accuracy, with exact (Clopper-Pearson) CIs
• New: roc() now gives the AUC with DeLong CIs and the Youden-optimal threshold

4. regtab() and survtab(): regression

• Fits multiple outcomes at once and returns a single publication-ready table
• Does adjusted risk ratios via log-binomial regression (with robust Poisson fallback), robust variance, and Firth regression for rare events
• New: survtab() for Cox models and hazard ratios
• New: autoplot() draws forest plots from any model

5. Reporting and export (new!)

• simtablr() builds Table 1 + Table 2 (crude and adjusted) as a single report which you can then send to word or excell or powerpoint
• as_methods() drafts the statistical methods paragraph from what you actually did with the variables you chose!
• export_docx() / export_xlsx() send everything to Word or Excel
• Journal styles (style("jama"), "nejm", "lancet") or your own with journal_style(), down to the Word table theme. Restyling never changes the underlying numbers and you can use the sabe object for new analysis, even after styling!

6. Statistical guidance (new)

SimtablR prints short notes (with references) when something deserves attention. Examples: an odds ratio with a common outcome, unadjusted p-values, or when the distribution of your variable violates an assumption from the model you're running. It should just tell you.

The package includes epitabl, a ynthetic cohort of 1,500 patients, so you can replicate the examples of the video

Links:
📦 Documentation: https://matheustg-14.github.io/SimtablR/
📄 Getting started: https://matheustg-14.github.io/SimtablR/articles/getting-started.html
🐛 GitHub (issues & feature requests): https://github.com/MatheusTG-14/SimtablR

Thanks to everyone who gave feedback on the first version! I’d love to hear what you think, what’s missing from your workflow, and any bugs you find on GitHub.

You'll see why I didn't spell out "AGD" when you look at the paper :)

While watching the Netflix documentary about plastics and fertility ("The Plastic Detox"), they had a shot of Dr. Swan's paper. The voiceover talked about how strong the relationship was between AGD and semen concentration.

Problem was that the scatter plot looked to me like there was hardly any relationship between those two variables.

So I went and looked at the paper (better format) and, while the R squared values for the plots in Figure 3 weren't given, it looked really low. Table 3 above claims a very low p value for the AGD and semen concentration.

image of plot and table

So the authors are claiming a relationship there and I'm not really understanding why the relationship doesn't show up in the scatter plot and why their beta coefficient and low p value does indicate a relationship.

Not well versed in statistics, so please ELI5.

Hi. I finished my BS-MS (integrated masters) majoring in maths and minoring in data science from one of the IISERs in India. I graduated this year. During my undergrad, I mostly had mathematics courses and my masters thesis was also in a topic related to number theory and my minor project was in statistical data science. I want go for PhD in statistics preferably in the US and my long term goal is jobs related to finance and quantitative research.

The list of course works I took:

Machine learning, real analysis, linear algebra, group theory, mathematical statistics, numerical analysis, scientific computing, complex analysis, Galois theory, topology, ODE, PDE, probability and stochastic processes, data structures, data analysis and visualization, measure theory, commutative algebra, manifolds, differential geometry, algebraic topology, functional analysis, number theory and cryptography, Fourier analysis.

My GPA is 3.97/4.00.

I have become genuinely interested in statistics through self-studying specially after I graduated and keeping in mind my long term goal, statistics suits more than pure mathematics.

I want to know which US universities should I apply whose PhD programme in statistics department will help me fulfil my long term goal and a tier list of the Unis I can get into (top schools, moderate and safe choices with the best ROIs) realistically specially coming from maths background.

Also, suggestions for unis in Europe/ Asia and masters programmes in mathematical finance/ applied maths with good ROI will be helpful. Any general advice will also be great.

Thank you.

I’m not really interested in becoming an MLE maybe BA or Data Science if my grades improve. I wonder in the age of AI if it’s worth learning how to code or if my time would be better spent just looking at AI code and seeing if it’s doing what I want. I seriously don’t want a job where I have to code frequently because it’s not my thing.

Interested in your thoughts

Edit:

To be clear: I don’t want to have a job where it’s coding intensive or I have to make something production ready. For business analyst or data analyst or aspiring product managers like is it still worth trying to learn to code given essentially “free” nature of code now

I saw this analogy made comparing AI finding proofs to open mathematical conjectures with AI constructing a model that finds a compound for treating cancer and statistically predicting that it would pass a Phase III clinical trial.

This seems like a problematic comparison to me because finding a proof to a conjecture is not the same as making a prediction with a statistical model. No matter how good a prediction is on paper, it is still no reason to treat that compound as though it had already passed Phase III (or even Phase I). On the other hand, a proof to a conjecture can be directly checked if it is right or wrong; there is no uncertainty to it. If the proof compiles on Lean, then it means that it is correct in as far as what being correct from compiling on Lean means.

Of course, an AI finding a compound that it predicts will pass a Phase III trial is still informative. It can motivate at least checking if the compound works invitro and if that checks out, one might be more motivated to proceed to a Phase I trial, but you would always still have to do the experiment, right?

Does this make sense or am I missing something?

For a while now, I've been thinking about the biostats MS, but from reading this and other subreddits, I realized the job market may be irredeemably horrible, that I need a Ph.D. for reasonable pay, that I should just get a stats MS instead, and what attracted me to was the application, but I'm reading that, while I understood it was more math than bio, apparently it's actually, like, 99% math and 1% bio. Is this an accurate understanding of the state of the biostats degree?

Edit: I should clarify that the math is fine to me, but I'm just less interested if, based on that 1% number, having a passion for observational research in health and bio is kinda irrelevant or unimportant for the job.

Hi everyone, I’m a solcal cc student majoring in Mathematics for Transfer with a 3.93 GPA, planning to transfer into Statistics. I'm targeting UCB, UCLA, UCD, Cal Poly SLO, and UW.

Long-term, my goal is to work on the project management side of quantitative and data-driven fields (e.g., Data Project Manager, Technical Program Manager, or Analytics PM).

My Background:

Academics: Currently in Calculus 3 (really enjoying it rn!). Preparing for Linear Algebra and Differential Equations.

Experience: 1 year of K-12 math tutoring; currently working as a college Supplemental Instruction (SI) leader for Calculus 2.

Certifications: Currently working toward a PMI Project Management certification (aiming for Agile/Scrum next).

My Questions:

1. As someone targeting data/statistics project management, how deep does my technical coding (R/Python) knowledge need to be compared to core statistics and PM methodologies?
2. I want to transition away from K-12 tutoring into something that builds both organizational and data-adjacent experience. What entry-level or campus roles should I look for?
3. How do UCB, UCLA, UCD, Cal Poly SLO, and UW compare in terms of industry connections, applied team projects, and opportunities to blend stats with project leadership?

Originally posted in r/poker, but I think this is also a good place for it, and there may be some poker enthusiasts here.

Vocabulary terms for people who may not play poker

Board - Community cards. The cards shared by everyone in a game line Holdem

Flop - First 3 community cards dealt

Nuts - best possible hand/hand of its type. So an Ace high flush on a board of King, 7 6 (all diamonds) is both the nuts and the nut flush

Broadway - a straight from the 10 to the Ace

Rainbow board - A board that has no more than 1 card in a single suit

Front door flush - A board that has 2 of a suit, and then the third in that suit is dealt making flushes possible

Backdoor flush - When a flush hits on a rainbow board, so the flop is 2 of clubs 3 of hears 7 of spades and the runout is X spades Y spades

1. I'm confused about what to do for my data. I can't use the common equal variance tests in my software because those are for one-way models only. May I ask for suggestions?

2. In JMP, I've discovered a heteroscedacity test (no name for the method, just heteroscedacity test) where it shows the variances of the two one-way models and the whole two-way model graphically with no p-value involved. However, some results suggest that variances are not equal in some one-way models, but when you look at the whole model, variances are equal. If this method is correct, should I just use the whole model graph when testing for the assumption for equal variances? Or should all the one-way models and two-way model show equal variances for the data to achieve the equal variance assumption?

3. Residuals are used in testing for assumptions, not raw data, right?

Suggestions are highly appreciated.

Hi everyone! I'm struggling with a statistical problem related to my experimental design, and I was hoping someone could help me.

I want to test whether the relationship between two traits, let's say height and weight, differs between two countries, for example, Japan and the USA. Standardized Major Axis (SMA) regression seems like a good approach, since I'm interested in comparing the slopes of the relationships between the two groups. However, my observations are not completely independent because individuals are nested within populations.

For example, in Japan, I sampled several individuals from population A, several from population B, and so on. I followed the same sampling design in the USA. Individuals from the same population are likely to be more similar to each other, so I need to account for this hierarchical structure. My initial approach was to calculate the mean of each trait for each population and then perform the SMA regressions using those population means. However, this approach removes a lot of the individual variation, which I would ideally like to retain. My question is: Is there a way to compare SMA slopes between two groups while accounting for the fact that individuals are nested within populations?

Hi! I’m an undergraduate student currently working on our research, and I’d like to ask for advice regarding our questionnaire.

Our questionnaire has two independent sections that measure different constructs:

Behavior - 5-point Likert scale

(Never-Always)

Perception - 6-point Likert scale

(Strongly Disagree-Strongly Agree)

These are separate constructs, and we are not planning to combine them into one overall score or compare/correlate the two scales. We simply want to analyze and interpret each section separately.

So my question is: is it statistically/methodologically acceptable to have different Likert scales in different sections of the same questionnaire, when the sections measure independent constructs and each section has its own separate computation and interpretation?

Thank you!

we have 1 day gap for my intro to stats class and it is GRUESOME - I am really bad at maths. Our professor barely teaches 1 problem and moves on and the problems we get in the exam are not straight forward, we have to figure out what test to use. We literally follow different formulas besides the one given in tbs so referring to textbooks for explanations is hard because they keep using their formulas to explain it making it all so confusing

The main issue is I am having trouble remembering the formulas, because every step has 1 formula, etc. (ie. ANOVA, T-test)

I have -
Chi-square test, Mann-Whitney U test, Wilcoxon Signed Rank test, Kruskal-Wallis test, Friedman’s test, Scatter plot; Correlation methods: Pearson’s correlation, and Spearman’s rank correlation, Z-scores; t-test – One-sample t-test, independent samples t-test, Paired samples t-test; One-way Analysis of Variance (ANOVA), Range, Quartile Deviation, Average Deviation, Variance, Standard Deviation,  – Bar graph, Pie chart, Line graphs, Histogram, Frequency polygon, Frequency curve, Ogive; Measures of Central Tendency: mean, median, mode – calculation

Literally any and all tips will be appreciated

Although I feel like a dummy when going back to relearn the basics, I do gain a different and deeper understanding of the formulas. Is it always going to be like this? Do you guys deep into your career still go back to the basics and deepen your understanding? Is it still like this 10 years later? Its my first time studying in uni and I dont like the idea that I have to sometimes just accept that I wont have full clarity over the mathematical tools im using, and that only later on the tools gain clarity. Sorry for any misspellings english is my third language.