Talk:Condition number

question about condition number

I'll move this to the reference desk in a day or two if someone isn't watching here.

Can someone help me with an estimate of how big a condition number actually has to be before a matrix is considered "ill conditioned?" I'm sure it's dependent on many things but I'd like to get a grasp on it.

I'm trying to show that in a bunch of cases (200 or so) two 2D vectors are "approximately" multiples of one another. So I make a 2x2 matrix out of these two vectors, and I find the condition number. On average the condition number is around 2e6, and the minimum is 9e4. Does this make the vectors approximately multiples of one another? moink 16:58, 7 Apr 2004 (UTC)

The error in the approximate solution is no greater than the (condition number) x (relative error in the initial solution). If the new approximate solution is not within the desired precision of the actual solution, then the system would be "ill-conditioned." Also, to obtain a higher precision in an ill-conditioned system, a much more accurate initial condition is required.Jaboles 20:49, 23 August 2006 (UTC)

It depends on how accurately you know these vectors. The amount of significant digits you lose is equal to log(condition number), so if you only have the vector components 3 digits accurate, then a condition number>1000 will produce garbage.131.155.215.153 (talk) 16:05, 7 September 2012 (UTC)

Confusing

"wheras a large condition number will enhance error in b." I think this is confusing because we are talking about the bad conditioning increasing the amount of error in the calculated x due do a small error in b (and not really doing anything to b). Richard Giuly 08:53, 1 November 2006 (UTC)

I agree. The next step is that you think of a better formulation and edit the article. These articles don't write themselves ;) Don't worry too much about making mistakes; they will be corrected. -- Jitse Niesen (talk) 10:44, 1 November 2006 (UTC)

I tried to change it myself. -- Jitse Niesen (talk) 11:20, 8 November 2006 (UTC)

I added a plausible derivation of the condition number for operator (induced) matrix norms. I hope it helps. Perdelsky 03:09, 5 August 2007 (UTC)

Relevance of precision

I am not sure if the relevance of finite/infinite precision is clear. If you have infinite precision, and do not assume any kind of errors in the input, the condition number is of no relevance. Berland 12:42, 18 January 2007 (UTC)

If you use exact arithmetic (infinite precision), and an iterative method to compute an approximation to your solution (whose rate of convergence degrades with increasing condition number), then the condition number is of great relevance. Lunch 17:38, 18 January 2007 (UTC)

You are absolutely right. I think these (subtle?) issues could/should be addressed in the article. Berland 21:22, 19 January 2007 (UTC)

---

"if the condition number is large, even a small error in b may cause a large error in x", is it just opposite? i think a large condition number means a small error in x cause a large error in b. Shangkun (talk) 11:35, 28 July 2008 (UTC)

Both are true. If the condition number is large, then a small error in b may cause a large error in x and a small error in x may cause a large error in b. That's because $\kappa (A)=\kappa (A^{-1})$. -- Jitse Niesen (talk) 13:17, 28 July 2008 (UTC)

Two problems with this article

And these they are:

1. The definition has a link to ill-conditioned that redirects to this article. So there is a need to explain what well-conditioned and ill-conditioned are and to remove this link.
2. The definition writes about problem's amenability to digital computation. However, in the section for condition number of a matrix it is mentioned that conditioning is a property of the matrix and not of floating point arithmetics or rounding off errors.

TomyDuby (talk) 02:07, 18 December 2008 (UTC)

The maximum value (for nonzero b and e) is easily seen to be the product of the two operator norms:

I don't find this so easy - please could somebody give more details about this line.Nlinton (talk) 13:18, 9 October 2009 (UTC)

Seconded. I'm mathematically literate enough to be interested in this article, but haven't been able to guess the principles that ought to be invoked to find this maximum. GRB, 15 August 2011. — Preceding unsigned comment added by 99.24.63.46 (talk) 13:54, 15 August 2011 (UTC)

If you are free to choose e and b such that the quantities are maximum, then ||A^(-1) e||/||e||=||A^(-1)|| and ||b||/||A^(-1) b||=||A||131.155.215.153 (talk) 16:13, 7 September 2012 (UTC)

It's obvious for the first part $\|A^{-1}e\|/\|e\|$ whose upper bound is $\|A^{-1}\|$ by definition (see induced matrix norm), not immediately for the second one though. Here's my attempt: First off, maximizing $\|b\|/\|A^{-1}b\|$ is equivalent to minimizing $\|A^{-1}b\|/\|b\|$. Or in other words, finding the smallest factor by which $A^{-1}$ stretches vectors. Since $\|A\|$ is the largest stretch factor for the mapping $A:x\mapsto b$, the smallest stretch for $A^{-1}:b\mapsto x$ consequently is $1/\|A\|$, and hence $\max _{b}\|b\|/\|A^{-1}b\|=\|A\|$. --138.246.2.58 (talk) 18:21, 7 February 2013 (UTC)

Make the change of variable $c=A^{-1}b$. Then $\max _{b}\|b\|/\|A^{-1}b\|=\max _{c}\|Ac\|/\|c\|=\|A\|$. 2607:F470:6:5002:5433:45C3:7BB5:97CB (talk) 01:56, 30 June 2014 (UTC)

Additions

I think it would be useful in the introduction to mention the condition number rule of thumb. It was mentioned to me in a Numerical Analysis class, and found that it cleared up some questions I had. The rule of thumb is if $\kappa (A)=10^{k}$, then you lose $k$ digits of accuracy on top of what would be lost to the numerical method due to loss of precision from arithmetic methods. This can be seen in Numerical Mathematics and Computing, by Cheney and Kincaid. What do you guys think? Mikesmith00 (talk) 01:16, 24 September 2010 (UTC)

Does Cheney and Kincaids rules of thumb apply in computations outside of the 2 norm? In the book they have it listed as $\kappa (A)=||A||_{2}||A^{-1}||_{2}$ And then they proceed to say that $k$ digits of precision lost resulting from the base ten logarithm of the aforementioned computation of the condition number.

There needs to be more clarification if this is not the case.

--Faraz.yashar (talk) 16:52, 20 November 2010 (UTC)

Mention 'error' or 'uncertainty' in early introduction

I think it would be very helpful for more general readers if the introduction (in particular the first paragraph---currently 2 sentences) mentioned that the condition number is generally used to evaluate how large the error/uncertainty of a function will be, given a certain error/uncertainty in the data/parameters. As it stands, the introduction is somewhat vague---while it is quite clear, *once* I know what its talking about. All Clues Key (talk) 00:16, 1 October 2012 (UTC)

NEEDS: Handling Ill-Conditioned Matrices

The phrase 'ill-conditioned' redirects here. I'm willing to wager that nearly anyone looking up 'ill-conditioned' is eager to find strategies for solving problems that have ill-conditioned matrices. I do not see anything useful presented here in that regard. I recommend that either (1) strategies for coping with ill-conditioned matrices be included in this page or (2) this issue get its own page. — Preceding unsigned comment added by 209.119.247.66 (talk) 18:33, 11 June 2014 (UTC)

What does it mean to be a multiple of one ?

"If the condition number is not too much larger than one (but it can still be a multiple of one)"

Which numbers would not be multiples of one ? — Preceding unsigned comment added by Clauariel (talk • contribs) 16:20, 17 April 2017 (UTC)

External links modified

Hello fellow Wikipedians,

I have just modified one external link on Condition number. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:

• Added archive https://web.archive.org/web/20070121001740/http://numericalmethods.eng.usf.edu/mws/gen/04sle/mws_gen_sle_spe_adequacy.pdf to http://numericalmethods.eng.usf.edu/mws/gen/04sle/mws_gen_sle_spe_adequacy.pdf

When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 18 January 2022).

• If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
• If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—InternetArchiveBot (Report bug) 00:49, 12 August 2017 (UTC)

What symbol?

What is the symbol for condition number? I recall being told—this recollection is not a high-quality citation—that, at least for a matrix of double derivatives, it is written |||H|||. The article writes it with a symbol in a low-resolution bitmap that might be an italic greek kappa (“κ(A) = 10^k”). Is it that? Is the triple-bar wrong? Please could the article say? JDAWiseman (talk) 10:58, 17 February 2020 (UTC)

Condition number and derivative

The condition number is an application of the derivative[citation needed], and is formally defined as the value of the asymptotic worst-case relative change in output for a relative change in input. The "function" is the solution of a problem and the "arguments" are the data in the problem. The condition number is frequently applied to questions in linear algebra, in which case the derivative is straightforward but the error could be in many different directions, and is thus computed from the geometry of the matrix. More generally, condition numbers can be defined for non-linear functions in several variables.

This paragraph has to be re-written by a specialist. This excerpt gives a false impression that the concept of the condition number revolves around the concept of derivative. Although there is an obvious connection between the two, neither is generalisation or the consequence of the other. I placed [citation needed], because the statement is meaningless (if understood in a broad sense) or technically false (if understood narrowly). AVM2019 (talk) 22:48, 27 February 2021 (UTC)

Here is a good possible lead for a future contribution, I quote Golub & Van Loan 4th ed. p.88:

... the condition number is a normalized Frechet derivative of the map A \to A^{-1}. Further details may be found in Rice (1966)

AVM2019 (talk) 22:57, 27 February 2021 (UTC)

Condition number of a rectangular matrix

"Matrices" section considers only square matrices. A request for extending this section, tidying it up and articulating clearly what is applicable to square matrices only. AVM2019 (talk) 23:01, 27 February 2021 (UTC)

Flawed definition – please someone fix it!

The definition at the very top ("General Definition..."), which involves the limit $\varepsilon \to 0$, is incomplete as is: one needs to explain how the numerator $\delta f(x)$ depends on $\varepsilon$. As it is currently written, $\delta f(x)$ does not appear to depend on $\varepsilon$ at all, which makes the ratio diverge in the limit. To fix, please specify how ${\tilde {f}}$ depends on $f$ and $\varepsilon$. — Preceding unsigned comment added by 2600:6C40:200:280:8817:9093:36AC:E742 (talk) 18:13, 11 January 2022 (UTC)

It is also not clear what $\delta x$ is. 103.249.233.200 (talk) 14:46, 1 June 2022 (UTC)

Yes, the "General Definiton" section is a disaster. I suggest to delete it and replace it with Condition_number#Several_variables, since it gives the needed general definition, as far as I can see.

I propose a new structure for the article:

• (0) Intro as it is;
• (1) General definition (move "Several variables" here)
• - (1b) Specialisation to scalar functions and examples (move the rest of "Nonlinear" here)
• (2) Specialisation to matrices
• - (2b) Specialisation to the matrix spectral norm (i.e., **the** matrix condition number; maybe also add the generalisation to rectangular matrices, see Trefethen&Bau p130 or Meyer p426)

But for now, we can just use the definition from Condition_number#Several_variables. AVM2019 (talk) 15:38, 1 June 2022 (UTC)