Hi! This is my second semester studying Statistics at a top university. However, I came from a poor high school education, and I wasn't used to studying regularly or keeping up with the university workload. As a result, I failed Calculus I.

I realized that I urgently need to change my study habits. I also want advice on how to do well in my degree and, most importantly, truly learn the subjects, especially considering the use of AI and other tools.

I would love to hear your thoughts on how I can become a better student. What study methods, habits, and strategies do you recommend?

Thank you!

I did a personal side project on why some years bring much worse smoke haze to Singapore. I'd like people who know stats to tell me where the design is weak before I show it to anyone in the field.

The chain I test, one observation per year:

1. Pacific El Niño index (June to August) vs Kalimantan rain (August to October), 1979 to 2025, n = 47. Pearson r = −0.79.
2. Sumatra rain regressed on the El Niño index and the Indian Ocean Dipole index, n = 47. I test the Dipole coefficient.
3. log10(fire detections) = a + b·rain + c·post2016 + e, one model per island, 2001 to 2025, n = 25. post2016 is a dummy for a fire-prevention policy shift after 2015. R² is 0.82 and 0.89.
4. Fire detections vs Singapore's seasonal mean PM2.5, 2016 to 2025, n = 10. Spearman rho = 0.88, bootstrap 95% CI 0.47 to 1.00.

I applied a Holm correction across these five tests, used 10,000 paired bootstrap resamples, ran leave-one-year-out, and swapped data sources as robustness checks. All five tests stay significant.

What I'm unsure about:

1. I wrote the analysis plan after I had already explored the same data. I say this openly, but is there anything more I should do besides waiting for future seasons as a true holdout?
2. The Sumatra fire model has a Durbin-Watson of 1.44. Should I switch to Newey-West errors or an AR(1) model, or is the bootstrap enough here?
3. Is OLS on log10 counts acceptable, or should this be a negative binomial GLM?
4. A dummy for "after 2015" vs a linear trend: both give the same sign. Is there a better way to handle a policy break with only 25 years?
5. With n = 10, is rho = 0.88 worth reporting at all, or should I present it as descriptive only?
6. The five tests form one causal chain, so they aren't independent. Is Holm the right correction, or is the correction itself the wrong framing?

Code, data and a 12-page write-up: https://github.com/Shelikeit69/haze-drivers-research

I built this with an AI coding assistant, which is part of why I want people to check the stats. Blunt answers welcome.

From what I understand, correlation only looks for a connection between two variables, while regression explicitly looks to see if one variable's increase/decrease positively or negatively impacts another variable's increase/decrease.

Hey, I have a decent background for distribution theory and statistical inference. I wanna get into random processes. I have no idea about the resources to start with. Can anyone suggest me some ?
Moreover, are there any discussion groups and channels on telegram ir any other platform specifically for this?

I’m considering taking a statistics course, and I’ve been thinking about the next steps—whether it’s significantly more beneficial for my career to pursue a technical specialization, a master’s, or even a PhD

Helloo:) I’m working on fatty acid composition analysis (from bird preen oil samples), with very little previous knowledge of statistics, and I’m looking for some help understanding what’s happening.
Background: I have a dataset with treatment group, sex, aviary of the sampled bird, and 10 fatty acids found in the preen oil samples. There are 3 treatment groups: germ-free, conventionalised, and parent-raised (although only the germ-free and conventionalised can be realistically compared). Between these 2 treatment groups, there are only a total of 27 samples, 13 in one and 14 in the other. The fatty acid cells contain proportions rather than absolute values (they always sum up to 1), so an example row looks like this: 

The main end goal is to see if the germ-free condition results in any changes to preen oil.

Now, for the analysis issues: I’ve computed robust Aitchison distances, which are supposedly a better choice for this data than Bray-Curtis(?) and ran a PCoA with them, followed by PERMANOVAs for treatment group, sex, and aviary (none of which were significant, unlike the marginally significant and high R2 results on B-C). 
I’ve also been advised to run some form of a linear model on each individual fatty acid, but I don’t quite understand where to start with that… Or which one would be suitable.
I would appreciate any feedback, comments, or advice on what I’ve done so far and what I’m trying to do!:)

I've spent the last weeks building a strict research pipeline for BTC/ETH spot trading (EUR pairs). Everything has been tested on historical data and in paper trading, with no real money yet. My methodology is probably stricter than most hobby setups. The result: nothing survives. I'd like to hear from people who got further, or who hit the same wall.

My setup (the rules I hold myself to)

• Criteria are written down and committed to git before each test and never changed afterwards.
• Costs are always included: 0.25%/side in early tests, now 0.10%/side (taker), plus slippage.
• Strict walk-forward. The result has to hold per year and per market regime.
• Placebo test: 1,000 block-bootstrapped fake price series. A strategy only counts as "proven" at p < 0.01.
• Bar: beat buy-and-hold after costs and have a smaller max drawdown, with 99% intervals and a Bonferroni-style correction for the number of attempts.
• A held-out "vault" dataset (Bitstamp 2018–Mar 2019) that nobody has looked at. Each strategy gets exactly one shot at it.
• A hard cap of 13 attempts in total. If nothing works by then, my conclusion is "not feasible at these costs".
• No LLMs in the trading decision. They have seen the future in their training data.

What I tested (all failed)

Other findings:

• 0 of 240 flexible-exit variants were positive, and 0 of 270 "conditions" from a trade journal were positive. The exit isn't the problem. The entry has no edge.
• Paper trading confirmed this. A dip rule went 10/10 winners in one night, but that was one single recovery move, not 10 independent trades. The backtest over 7 years says it's placebo-level.
• After FOMC decisions crypto volatility roughly doubles in the following hour (BTC 0.66% → 1.25%), but the direction is unknown, so there's nothing tradeable.
• Lead-lag between exchanges: the lead I measured is ≤ 0.06%, far below the cost of a round trip.
• With ~0.2–0.5% friction per round trip, even a near-perfect predictor at horizons under 1 hour earns almost nothing.

Where I'm stuck: my real questions

1. Source of edge. For a retail trader without colocation or speed, what edge in crypto spot is documented to survive 0.1–0.2% costs? Is it structural (liquidation cascades, funding imbalances, forced selling, order book imbalance) rather than price patterns?
2. Is my bar too strict, or the right one? p < 0.01 against block-bootstrap placebos, plus beating buy-and-hold on Sharpe and drawdown. How do serious shops set this bar for single-asset strategies?
3. Beating buy-and-hold in crypto. BTC/ETH buy-and-hold has a huge drift. Should a timing strategy be benchmarked against risk-adjusted buy-and-hold, or against cash?
4. Regime dependence. Many strategies seem to work in one regime only. Is it legitimate to run a regime-switching approach, or is that overfitting in disguise?
5. Paper vs live. How far did your paper results drift from live (fills, slippage, latency)?
6. Has anyone gotten past this point? Not "my backtest is great", but sustained live profit after costs on BTC/ETH spot. What was the edge, and how long did it last?

What's running now: paper trading 24/7 on real exchange data, with 42 strategies in a "sandbox" that each have a pre-set required number of trades and a deadline. I'm also collecting order book, liquidation and lead-lag data for months before forming new hypotheses.

I'm happy to share the methodology. Honest stories about what didn't work for you are especially welcome. That's the rarest information online.

I am performing an ordinal regression analysis reduced to 10 predictors from an original set of 50. Sample size: N = 380. One of these predictors is the primary predictor of interest (Safety Hazards [SH], a continuous variable). The rest are potential confounders of the primary predictor or contributors to the model.

My primary outcome is Job Satisfaction (JS).

I distilled the 10 candidates from the oriwith 10 predictors, reducedginal 50 by running ordinal bivariate regressions on each candidate.

a) A crude model [JS <-- (beta)*SH]

b) 49 adjusted models: [JS <-- (beta)*SH + any of the 49 predictors].

1. If any of the bivariate regressions produced a CIE of 10% or more for the primary predictor, SH, it made the list of 10, regardless of its p- value.

2. If any of the 50 candidates had a p- value of < = 0.1, regardless of its CIE value, it made it to the list of 10 as well.

I sort the list of 10 by CIE value and use that order to determine which variable to test next in the model.

I then begin the final fitting process to extract the most plausible confounders of SH and true model contributors from the 10 predictors. I use 2 different regression methods: Forward Stepwise Regression (FSR) and Backward Stepwise Regression (BSR).

The conditions to stay in the model are: If the p-value > 0.1 AND CIE <10%, then reject from the model; otherwise, retain.

In FSR, I start with a sparse (crude) model and sequentially add a predictor, then test whether it meets the retention criteria. If so, I retain it, add the next predictor to the new model, and apply the same retention/rejection criteria until all predictors have been tested.

In BSR, I start with a full model and sequentially remove predictors, one at a time, and perform similar rejection/retention tests as in FSR until all predictors have been tested.

However, I find that the FSR and BSR algorithms retain and reject different variables. To be clear, about 60% of variables are common to both algorithms.

My question is: why do FSR and BSR algorithms produce different results?

What is different about starting with a full model (BSR) versus a sparse model (FSR)?

What is different about starting with a sparse model (FSR) versus a full model (BSR)?

Please try to give an explanation that does not involve the term 'variance', as I tried to explain to my PI, only to be told that is the wrong way to explain it.

Thanks

David

greetings everyone, I am a grad student from forestry background and i am doing a research project on biomass estimation via sentinel 2 variables. I have 47 plots with field measured biomass value and i ve analyzed 18 different remote sensing variables that acts as the predictors of biomass. predictors correlated well with the measured biomass but the regression model stated lower R square value (i.e. 0.40) at which isnt like i expected. I ran five different models MLR, PLSR, RF, GBR, linear SVR, Ridge (im a beginner in coding and all, so claude helped me) etc but neither improved the coeft of determination. I followed the exact methodological framework from the literature but lost somewhere. My main objective is to publish the paper but the lower accuracy is becoming the problem. Sorry if i couldnt explain it clearly, feel free to ask via comment or dm/ any suggestions and feedbacks are appreciated. Thankyou!

Hello!

I have minor revisions on my scientific paper. One of reviewers comment was: "please include the statistical model equation in the Methods section.". My doubt is: I performed a Beta Regression Model in R: model_beta <- betareg(proportion ~ plant + week, data = plants).

Is this what I´m supposed to add to the paper? Im a biologist, not a statistician, so im not sure what to write ... Thanks!

Hi everyone, just want to preface by saying i am very much a microbiologist and know extremely little in the world of stats. I’ll give some context to my experiment: I preformed an enrichment where i grew bacteria in vials (n= 3) for 6 weeks, then took a portion of those growths and added it into new vials (n=3) for another 6 weeks. The vials let’s say A1, B1, C1 corresponded to the second batch of vials A2, B2, C2 where in the samples from A1 was added to A2 etc. I did this at two different temperatures (20 and 30 degrees). So i have two variables, the time and the temperature, giving me four groups.
Group 1. 20 degrees celsius week 6 (rep1, rep2, rep3)
Group 2. 30 degrees celsius week 6 (rep1, rep2, rep3)
Group 3. 20 degrees celsius week 12 (rep1, rep2, rep3)
Group 4. 30 degrees celsius week 12 (rep1, rep2, rep3)

I have species level relative abundance data from shotgun metagenomic sequencing and want to determine whether community composition differs between the four groups and ideally a post hoc specific to which. The complication is that at each week 12 culture was subcultured from its corresponding week 6 culture with temperatures maintained. Therefore, week 6 and week 12 are linked observations rather than independent samples. There are three lineages per temperature. I’ve tried Bray curtis PERMANOVA in R using vegan::adonis2() with restricted permutations but the permutation schemes i’ve tried don’t test the temperature, week and interaction effects as intended. I also found significantly different multivariate dispersion between the temperature groups.

So my question is:
1. What statistical method would you recommend for testing community composition while accounting for the linked time points?
2. Is there an appropriate permutation scheme or alternative model for testing temperature, time, and their interaction with this design?
3. What methods would be suitable for identifying differentially abundant species from these metagenomic data?

I’m looking for a statistically defensible approach used in similar microbial enrichment or longitudinal studies. References to relevant papers or example R code would be greatly appreciated. Thank you!!!

Context: I have to send my final dissertation manuscript to my committee tomorrow. My chair called me this morning and told me my stats are all wrong because I ran too many tests with too small of a sample. When I asked for guidance on resolving this, they said "You want a PhD? Figure it out". I have spent all morning reading resources and I have gotten myself more confused.

This is a single group pre-post design based on feasibility and exploring clinical outcomes. I originally had planned paired samples t-tests, but I ended up with such a small sample that it would inflate error rate. Also because of the small sample, any assumptions of underlying distributions can't be met, so I'm using exact sign tests. I modeled my stats after another published pilot study with 7 participants and 15 comparisons. They do not do a correction.

I ran the 7 exact sign tests for my outcome variables, found significance in 6/7, and also presented the data visually. If I did a Bonferroni correction to control for multiple comparisons, my new p-value would be .007 and basically impossible to get with exact sign tests. Is my only option to only present stats for my main variable, and only present descriptive/visuals for the rest?

Suppose two forecasting models have nearly identical average prediction errors, but those errors evolve very differently over time.

In one model, errors are approximately random. In the other, they form persistent clusters, oscillations, or systematic transitions.

Could the temporal structure of prediction errors provide an early signal of changes in the underlying system, even before conventional change-point detection identifies a regime shift?

I'm particularly interested in whether anyone has investigated this using autocorrelation, entropy, or multiscale analysis of prediction errors.

What statistical test would you consider most convincing for evaluating this hypothesis?

I had this idea based on something I've noticed: people often assume that others think the same way they do, and that their own opinions reflect what the majority believes.

So I thought, why not turn that into a game?

The idea is to test how well you can predict what the majority actually thinks, and see whether your own opinions are as popular as you might expect.

One thing I’m still unsure about is how many players would be needed to get a statistically meaningful representation of the population for each question. I don’t really know much about statistics, but it would be interesting to figure this out so I can estimate how many players are needed for the game to work properly. 🤔

Was he into older sisters, like onee-san? ._.

From my knowledge I don’t think there is a solution to this problem in the current scenario, but i am hoping somebody will have some useful input.

So I have a data matrix, which is similiar to a spring balance weighing design matrix(probably I should not call it that, since no design was involved) basically the variables take binary outcome 1,0 based on if they are present in the machine or not, and i have a final response variable which is the total weight of the machine. Now my goal is to estimate these individual weights of the components.

The dimensions of the matrix is 11268*408 , where 407 of the columns are the components.

There are quite a few redundant rows as well, their total weights sometimes differ by 30-40kgs. In hope of making the problem easier i have replaced them with one row and averaged the weight.

The matrix is unfortunately singular(which is why i am here ig), and with some level of threshold on the svd Values, i have found that it has a rank of 349 and a nullity of 61. Due to this there are variables and combinations which are not estimable, at least in the linear sense. So i am not sure if going forward with a pseudo inverse based least squares or anything would work.

I am sorry if there question and my framing with the theory is not clear, its been some time since i am out of touch.

Is there someway i can think of solving this problem? Even if some of it can be solved.

What is everyone's experience with Measure Theory during their phd?

For context, I'm a phd student in statistics taking measure theory and I'm experiencing burnout for this course. Dont get me wrong, I love the materials and the way it has reshaped my thinking of probability, but the class is genuinely so hard that class average was 50% for our first midterm. For the HW questions, I'm constantly struggling and sometimes it takes me 2-3 hours to solve a question that is 1 line long. I do not come from a pure math background and did not take Real analysis in school so I wonder if that's also a reason I'm having such a hard time.

Everyone I met makes this class sound like it's the most important class I will ever take my entire life so it makes me feel even worse that I'm struggling so much. On top of that I spend upwards 30 hours a week trying to study for it which doesn't help. I think I've accepted the difficulty of the subject, but I'm worried about the burnout - any other duties I have, I have to wait until the day of to do them because all I do everyday is study measure theory. I've already cut down on my social life (I only hang out with friends for 1 night (maybe dinner + drinks) every 2 weeks) to have some down time for myself during the weekends.

Would greatly appreciate any advice! TIA.

Thank you all I’m getting through these comments

Hello, this is my third attempt at trying to pass statistics in (college). And every professor is different, so there are different things expected of me. This professor she gives us 60 question quizzes with only 60 minutes to get through. And it’s very hard because all of her questions are word problems.

These pictures are some random ones. I took a picture of so that I could show my Tutor. I have been going to a Tutor, but I still don’t understand how to apply the terms and how each mechanism works. For example, I don’t understand why when Alpha level is decreased, the common zone increases. And I don’t know if that means to reject the null or to fail to reject the null?

I’m not really sure how to interpret data and what it would mean especially on these multiple questions quizzes. I’ve read the textbook and take notes and all that, but I still don’t really understand how all of these things come together? I know what they are individually.

I get most of the problems right on worksheet, so I’m not sure why I keep failing quizzes. I’ve only passed two of the five I’ve taken. And I happened to do well on my midterm, but I at an F in the class.

Hello everyone. I'm in quite a pickle. I want to analyse my data, but I really don't know enough about statistics. I should've thought about this beforehand. Either way, I have survey data, where I have the following:

- 30 pictures which are independent from each other

- I have asked people to assess how realistic, AI-generated, and how well it fits a certain theme in a 7 point Likert scale.

- Each participant evaluated every picture, so I have a "seperate dataset" on how realistic, AI-generated and how fitting the pictures are

- I can see what the highest means are, but I want to determine per category which picture is rated the highest/lowest, also when taking into account things like spread

How do I do that? I feel like I have too many categories

I have a random question/situation. I was speaking to someone who runs a business, and they have massive data sets -- it's athletics based, but you would have literally millions of data points, and presumably a ton of subsets.

Aside from GAMBLING -- please -- are there any academia uses or a/b tests that would make that data useful. I guess, when he said a professional company's data set is worth more than it's tv rights, it got me thinking that potentially there is a way to use his data in statistical testing that makes it valuable.

Does this make sense? Like are there cross-professional or random data sets that are valuable outside of their intended specific use?

I would really appreciate some help to know:
(1) if patients within the same clinic are correlated (ICC ~0.04), will adding more patients to the SAME 4 clinics help? Or does it only matter if we increase cluster NUMBERS?

(2) Is it normal to have unequal totals between the two arms?

Context: We are running a 6 month pilot feasibility cluster RCT in endocrinology clinics for a diabetes intervention (the intervention just tests a new workflow with the addition of a new profession + better communication channels). We randomized 20 clinics; 4 agreed to deliver the intervention, and 10 agreed to be in the control (usual care; Zelen's design). We are all clinically trained (pharm, med) with zero prior clustering experience. We are not powering for efficacy or effectiveness, we are only powering for feasibility

Some on our team think: ok since we only have 4 intervention clinics, let's just recruit 20–30 patients per clinic to have a better sample size. While the PI says his sample size calculation shows that 8 patients per intervention clinic (and 5-6 per control) is enough and anything else is just more headache without benefit.