What is Likelihood: Definition and 139 Discussions

In statistics, the likelihood function (often simply called the likelihood) measures the goodness of fit of a statistical model to a sample of data for given values of the unknown parameters. It is formed from the joint probability distribution of the sample, but viewed and used as a function of the parameters only, thus treating the random variables as fixed at the observed values.The likelihood function describes a hypersurface whose peak, if it exists, represents the combination of model parameter values that maximize the probability of drawing the sample obtained. The procedure for obtaining these arguments of the maximum of the likelihood function is known as maximum likelihood estimation, which for computational convenience is usually done using the natural logarithm of the likelihood, known as the log-likelihood function. Additionally, the shape and curvature of the likelihood surface represent information about the stability of the estimates, which is why the likelihood function is often plotted as part of a statistical analysis.The case for using likelihood was first made by R. A. Fisher, who believed it to be a self-contained framework for statistical modelling and inference. Later, Barnard and Birnbaum led a school of thought that advocated the likelihood principle, postulating that all relevant information for inference is contained in the likelihood function. But in both frequentist and Bayesian statistics, the likelihood function plays a fundamental role.

View More On Wikipedia.org
Recent contents Top users
  1. Q

    Proper likelihood function of the ratio of two spectra

    Hello PF'ers, I am doing an unbinned likelihood analysis where I am analyzing the ratio of two spectra: \[ \frac{S_{1}(E)}{S_{2}(E)} = F(E) \] and each spectra, \[ S_{1}, S_{2} \] has its own data set. My first idea was to take the function, \[ F(E) \] and divide by the integral of...
  2. D

    MHB What is the Maximum Likelihood Estimator for Uniform Distribution Endpoints?

    I need help on this problem, anyone know how to do it? Suppose you have n independent observations from a uniform distribution over the interval [𝜃1, 𝜃2]. a. Find the maximum likelihood estimator for each of the endpoints θ1 and θ2. b. Based on your result in part (a), what would you expect...
  3. F

    MHB Show that the maximum likelihood estimator is unbiased

    Consider a density family $f(x,{\mu})=c_{{\mu}}x^{{\mu}-1}\exp(\frac{-(\ln(x))^2)^2}{2}$ , where $c_{{\mu}}=\frac{1}{{\sqrt{2{\pi}}}}\exp(-{\mu}^2/2)$ For a sample $(X_{1},...,X_{n})$ fnd the maximum likelihood estimator and show it is unbiased. You may find the substitution $y=\ln x$ helpful...
  4. P

    Finding maximum likelihood estimator

    Homework Statement The independent random variables X_1, ..., X_n have the common probability density function f(x|\alpha, \beta)=\frac{\alpha}{\beta^{\alpha}}x^{\alpha-1} for 0\leq x\leq \beta. Find the maximum likelihood estimators of \alpha and \beta. Homework Equations log...
  5. I

    How Do You Compute the MLE for p in a Treatment Effectiveness Study?

    Homework Statement 1. An experiment consists of giving a sequences of patients a risky treatment until two have died, and then recording N, the number who survived. If p is the proportion killed by the treatment, then the distribution of N is: P(N=n)=(n+1)(1-p)n p2 1)Find a general formula...
  6. E

    Kalman Filter EM decrease log likelihood

    Hi all I am using the Kalman Filter with an EM algorithm, (schumway and stoffer). From my understanding the log likelihood should monotonically increase. In some instances however I obtain a decrease in the log likelihood, What can I infer from this? Has anyone experience of these...
  7. S

    Question to Maximum Likelihood

    Hello, I have a question to Maximum Likelihood Estimation. The typical form of MLE looks like this: X = Hθ + W. W is gaussion with N(0, C). θml = (HTC-1H)-1HTC-1X I think θml can only be calculated after a lot of measurements are made, that is, there are plenty of samples of H and X...
  8. D

    Likelihood of pairs in a range?

    Running a computer script (included below) I was testing to see how long it would take to match two numbers when selected at random from within a range. To my surprise the percentage of possibilities explored before finding a correct answer decreased as i raised the range. Is this correct? It...
  9. G

    Maximum Likelihood Estimator Problem

    Homework Statement 1. Suppose the data consist of a single number X, and the model is that X has the following probability density: f(x|θ) =(1+ xθ)/2 for -1≤ x ≤1; =0 otherwise. Supposing the possible values of θ are 0 ≤ θ ≤ 1; find the maximum likelihood estimate (MLE) of θ, and find...
  10. Jameson

    MHB Log likelihood function and EM algorithm

    I'm going to start by asking about an example from class and then hopefully use that to work on the problem I need to solve. Here is an example: Let's say we have a multinomial distribution $x \sim M(n;.5+.25\theta,0.25(1-\theta),0.25(1-\theta),0.5\theta)$. The likelihood function of $\theta$...
  11. T

    Measuring Membership, or Likelihood, of given Point

    Hello: I've three circles as seen in the image. Source: http://www.picpaste.com/Quantify-PwdfQxLB.png Within the innermost circle is a point. I'd like to determine the likelihood, or probability, that this point belongs to the innermost circle, the middle-circle, and outermost...
  12. V

    Likelihood Ratio Statistic & P-value

    Homework Statement Homework Equations So far I have only worked on question 1, as I was not able to solve it. The likelihood ratio test statistic is defined as follows: λ = 2 Log(L(theta-hat)/L(theta-hat_0)) Where L is the likelihood function, the product of all the pdfs/pmfs, and theta-hat is...
  13. M

    EM algorithm convergence KF log likelihood decrease

    Hi everyone, Im running the KF to learn parameters of a model, the log likelihood of the p(Y_{k}|Y_{k-1}), however decreases. Can anyone advise, does this mean my implementation is wrong or can this just be the case. Advice appreciated Thanks
  14. J

    Likelihood function of a gamma distribution

    Homework Statement Hi all, I missed the day of class where we went over likelihood functions, and I'm quite confused! For example, let's say I have n Xis, where each Xi ~ Gamma(a,b), where a and b are unknown. I want to find the likelihood function of a and b, but I don't think I really...
  15. Y

    MLE Homework: Distance Measurement with Odometer Error

    Homework Statement John wants to measure the distance from his home to his office, so he drives to work several times and measures the distance on his car's odometer. Unfortunately, the odometer records distance only to the nearest mile. (Johns odometer changes abruptly from one digit to the...
  16. S

    Hidden Markov Models - Likelihood of

    Hi, I try to teach myself Hidden Markov Models. I am using this text "www.cs.sjsu.edu/~stamp/RUA/HMM.pdf" as Material. The Introduction with the example was reasonable but now I have trouble in unterstanding some of the derivation. I can follow the math and use the formulas to get...
  17. C

    Statistical models and likelihood functions

    Homework Statement I have a couple of notation interpretation questions: 1) What does f_X(x|θ) represent in this case? The realization function of of our random vector X for some value x and a parameter θ (so that if our random vector has n random variables, its realization vector will be a...
  18. C

    Likelihood Functions: Parameters & Probabilities

    As far as I know, the definition of likelihood functions is the probability of a given random variable result given some parameter (please correct me if I'm wrong). What kind of parameters are usually handled by likelihood functions? Population parameters? Statistical model parameters? Both?
  19. S

    Maximum Likelihood Estimator + Prior

    1.Suppose that X~B(1,∏). We sample n times and find n1 ones and n2=n-n1zeros a) What is ML estimator of ∏? b) What is the ML estimator of ∏ given 1/2≤∏≤1? c) What is the probability ∏ is greater than 1/2? d) Find the Bayesian estimator of ∏ under quadratic loss with this prior 2. The attempt at...
  20. S

    How to Find Maximum Likelihood Estimators for Sample Data?

    Homework Equations L(x,p) = \prod_{i=1}^npdf l= \sum_{i=1}^nlog(pdf) Then solve \frac{dl}{dp}=0 for p (parameter we are seeking to estimate) The Attempt at a Solution I know how to do this when we are given a pdf, but I'm confused how to do this when we have a sample.
  21. D

    AICc and parameter likelihood from repeated fits

    I'm looking at using AICc to compare a set of nested models and wondered about the following. First, my the errors in my n data points around the fit are not i.i.d. Gaussian (the data have an artifact that the models will not able to fit easily and the artifact introduces dependencies and a...
  22. P

    Comparing Least Squares and Maximum Likelihood?

    Hi, Below is my attempt at a comparison between the two above-mentioned methods of estimation. Does anything in the table lack in validity and/or accuracy? Should any properties, advantages/disadvantages be eked out? Any suggestions/comments would be most appreciated! MLE...
  23. A

    What is the Probability of Multiple Universes?

    There are likely to be infinite, or at least countless universes out there. We haven't seen them because any signs of their existence, such as wavelengths along the EMS, may very well take trillions of years go get here, and also may very well be too faint to detect from the astronomical (excuse...
  24. M

    Bayesian method vs.maximum likelihood

    Hi, Wondering if there is any priorities one method has versus the other one and are there any specific cases where to use one vs.other? regards
  25. B

    Proving sufficiency via likelihood functions

    Homework Statement Let Y1,Y2,...,Yn denote independent and identically distributed random variables from a power family distribution with parameters α and θ. Then, if α, θ > 0, f(y|α, θ)={αy(α-1)/θα, 0≤y≤θ; 0, otherwise. If θ is known, show that ∏i=1n Yi is sufficient for α. Homework...
  26. K

    Inferential statistics-maximum likelihood function

    Homework Statement suppose that n cylindrical shafts are selected at random from the production of the machine and their diameters and lengths are measured. it is found that N11 have both measurements within the tolerance limits, N12 have satisfactory lengths but unsatisfactory diameters...
  27. K

    MLE of μ for X1,...Xn with Known σ2i

    is it possible to estimate all parameters of an n-observation (X1,...Xn) with same mean, μ, but different variances (σ21,σ22,...,σ2n)? if we assume that σ2i are known for all i in {1,...n}, what is the mle of of μ?
  28. S

    Schools Likelihood to enter foreign universities for Physics Ph.D

    Hello, I'm currently studying under an integrated M.S. program, majoring in Physics (Math as additional subject), and later wish to do Ph.D in theoretical Physics (broadly in particle physics). I know that universities look at factors like GRE score, reference letters, GPA, etc. as criteria...
  29. Q

    Likelihood of inheriting autosomal dominant diseases

    This IS wrong. This only holds true if one of your parents is HETEROZYGOUS dominant. Hh x hh => 50% chance of offspring having Huntington's disease. However, if one of your parents is HOMOZYGOUS DOMINANT .. HH x hh => 100% chance of offspring having Huntington's diseases. Simple Punnett...
  30. G

    Finding maximum likelihood function for 2 normally distributed samples

    Homework Statement The question is about how to combine to different samples done with 2 different methods of the same phenomena. Method 1 gives normally distributed variables X_1,X_2,...X_{n_1}, with \mu and \sigma^2_1 Method 1 gives normally distributed variables Y_1,Y_2,...,Y_{n_2}...
  31. B

    Maximum Likelihood Estimator, Single Observation

    Homework Statement An observation X has density function: f(x,/theta)=6x/(t^3)*(t-x) where t is a parameter: 0<x<t. Given the single observation X, determine the maximum likelihood estimator for t. Homework Equations Included below The Attempt at a SolutionFor a sample size of n...
  32. B

    What Is the Maximum Likelihood Estimator for t Given the Observation X?

    Homework Statement An observation X has density function: f(x,/theta)=6x/(t^3)*(t-x) where t is a parameter: 0<x<t. Given the single observation X, determine the maximum likelihood estimator for t. Homework Equations Included below The Attempt at a Solution For a sample size of...
  33. C

    Find Maximum Likelihood Estimator of Gamma Distribution

    Given f(x; β) = [ 1/( β^2) ] * x * e^(-x/ β) for 0 < x < infinity EX = 2β and VarX = 2(β^2) Questions: Find the Maximum likelihood estimator of β (I call it β''), then find Bias and variance of this β'' 1/ First, I believe this is a gamma distribution with alpha = 2. Is that right? 2/...
  34. K

    Statistical Analysis - Maximum Likelihood Fit

    Homework Statement I have a set of data from the DAMA experiment in which a detector attempted to measure collisions with 'WIMP's [Weakly Interacting Massive Particles] as a candidate for dark matter. The detector records the time in days of a collision event. After binning the data and...
  35. Z

    Stats question- likelihood function for a ratio

    Homework Statement According to genetic linkage theory, observed frequencies of four phenotypes resulting from crossing tomato plants are in the ratio 9/16 + a : 3/16 - a : 3/16 - a : 1/16 + a. In 1931, J.W. MacArthur reported the following frequencies: Observed...
  36. S

    Linear regression and maximum likelihood estimates

    Homework Statement Suppose that data (x1,y1),(x2,y2),.?.,(xn,yn) is modeled with xi being non random and Yi being observed values of random variables Y1,Y2,...Yn which are given by Yi = a + b(xi-xbar) + σεi Where a, b, σ are unknown parameters and εi are independent random variables each...
  37. P

    What is the Interpretation of Log Likelihood in Molecular Data Analysis?

    Hello, I am a Bio informatician and encountered Likelihood while executing the Molecular data. I have used one software that is using the Hidden Markov Model in addition to the EM Algorithm and Viterbi algorithm. After calculations are done already, in addition to the output, it is giving me...
  38. A

    Determining Likelihood of Divergance

    Currently utilizing very simple logic in determining the directional trend of a signal line, and was hoping someone might be able to offer a suggestion as to a more effective method of filtering false signals. As it stands the logic being used for determining the direction of a trend is if...
  39. H

    Schools Post-GRE assessment of grad school acceptance likelihood

    Background: I'm a senior at a large state university studying electrical engineering, and I will graduate in May. Currently, my GPA stands at 3.08. I am on track for a 4.0 this semester, which includes a graduate-level course in digital communications systems. By the time I graduate, I should...
  40. T

    Distribution of Maximun Likelihood Estimator

    Hey guys how are you? I have the following question: Let X1,X2,...,Xn be a random sample from a Pareto distribution having pdf f(x|b)= (a*b^a)/x^(a+1) where x>=b (1) Determine the maximum likelihood estimator for b, say b' on (0,infinity) and by considering P(b'>x) or otherwise show...
  41. I

    Bayesian inference of Poisson likelihood and exponential prior.

    Hey I have some problems understanding my statistics homework. I am given a data set giving the number of calls arriving to different switchboards in three hours as well as the total phone call duration in minutes for each switch board. Something like i y_i t_i -------------- 1...
  42. C

    Why is likelihood function defined as such?

    Hi everyone, This is not a homework question but something I thought of while reading. In the method of maximum likelihood estimation, they're trying to maximize the likelihood function f(\vec{x}| \theta ) with respect to \theta. But shouldn't the likelihood function be defined as...
  43. M

    Monotone Likelihood Ratios-Most Powerful Test

    Monotone Likelihood Ratios--Most Powerful Test This isn't really a problem, its more of a theory question. I'm having trouble understanding the reading in my textbook (Introduction to Mathematical Statistics, by Hogg Craig, and McKean). I'm looking at the section of using concepts dealing...
  44. G

    A simple baysian question on likelihood

    Hi all, In this question, I found that \Pr(X_i|\theta)=\frac{\exp(\theta x_i)}{1+\exp(\theta)} and I carry on with the likelihood being \frac{\exp(\theta \sum x_i)}{(1+\exp(\theta))^n} and so s=\sum x_i = Tn I need some help with part (c)...
  45. M

    News What is the likelihood of an EMP attack?

    I was wondering -- what are the chances of someone getting away with an EMP attack? What measures is the US taking to prevent such an attack? The way I see it there are three scenarios that are more likely to happen: a) the attacker attempts to fly into the United States, the US sees an...
  46. P

    How Do You Calculate Maximum Likelihood Estimates for Different Distributions?

    I actually have two questions, both of which are on the same topic Homework Statement Consider X = number of independent trials until an event A occurs. Show that X has the probability mass function f(x) = (1-p)^x p, x = 1,2,3,..., where p is the probability that A occurs in a single trial...
  47. P

    Solving Likelihood Questions in Soccer: Team A and B

    Homework Statement Hi everyone, I'm working through some max likelihood questions and am badly stuck on this one. Please could you take a look at what I'm doing and tell me if I'm going in the right direction? Q. Team A and B play two games of soccer, each game having two halves of equal...
  48. P

    Maximum Likelihood Estimator Question

    Homework Statement A bag contains sequentially numbered lots (1,2...N). Lots are drawn at random (each lot has the same probability of being drawn). Two lots are drawn without replacement and are observed to be X_1 = 17 and X_2 = 30. What is the MLE of N, the number of lots in a bag...
  49. P

    Maximum Likelihood Estimator Question

    Homework Statement Lifetimes of components are Gamma distributed. The parameters of the Gamma are shape = a scale = λ The pdf is: f(x) = (λ^a).x^(a-1).e^(-λx)/Γ(a) In this case, it is known that a = 3. Obtain the MLE of λ. Homework Equations The Attempt at a Solution Hi...
  50. S

    Maximum likelihood estimator and UMVUE

    Homework Statement Let X_{1}, ... , X_{n} be a random sample from f\left(x; \theta\right) = \theta x^{\theta - 1} I_{(0, 1)}\left(X\right), where \theta > 0. a. Find the maximum-likelihood estimator of \theta/\left(1 + \theta\right). b. Is there a function of \theta for which there...
Back
Top